Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-3117

[RHCOS 9] Collector pods throwing CreateContainerError in RHCOS 9 cluster

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • NEW
    • VERIFIED
    • Log Collection - Sprint 225, Log Collection - Sprint 226

      Version:
      installer_payload_image: registry.ci.openshift.org/rhcos-devel/ocp-4.12-9.0:4.12.0-ec.3
      Logging: 5.6

      Steps to reproduce:
      1. Install logging operators on RHCOS 9
      2. Deploy CL using vector/fluentd

      cluster-logging-operator-76c976d787-wdcgm       1/1     Running                0          5h25m
      collector-87plr                                 1/2     CreateContainerError   0          2m42s
      collector-c8q5k                                 1/2     CreateContainerError   0          2m42s
      collector-cctkz                                 1/2     CreateContainerError   0          2m42s
      collector-tqvnt                                 1/2     CreateContainerError   0          2m42s
      collector-ts6zc                                 1/2     CreateContainerError   0          2m42s
      collector-wq8xc                                 1/2     CreateContainerError   0          2m42s
      elasticsearch-cdm-hw29zgc4-1-69786f79d9-sqtr6   2/2     Running                0          101m
      elasticsearch-im-app-27737745-zb7hd             0/1     Completed              0          3m46s
      elasticsearch-im-audit-27737745-vrfc5           0/1     Completed              0          3m46s
      elasticsearch-im-infra-27737745-5xgdz           0/1     Completed              0          3m46s
      kibana-5bd795c9c-jg8cf                          2/2     Running                0          101m

      Seeing below error in collector pods:

      Events:
        Type     Reason          Age                    From               Message
        ----     ------          ----                   ----               -------
        Normal   Scheduled       4m52s                  default-scheduler  Successfully assigned openshift-logging/collector-87plr to ip-10-0-217-251.us-east-2.compute.internal by ip-10-0-129-208
        Normal   Started         4m51s                  kubelet            Started container logfilesmetricexporter
        Normal   AddedInterface  4m51s                  multus             Add eth0 [10.130.0.90/23] from openshift-sdn
        Warning  Failed          4m51s (x2 over 4m51s)  kubelet            Error: container create failed: time="2022-09-27T07:46:06Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Normal   Pulled          4m51s                  kubelet            Container image "registry.redhat.io/openshift-logging/log-file-metric-exporter-rhel8@sha256:2e143b2fd76e2d24c8368a4f9a0e492b4c19811403a4665b676fe520edd2a1b1" already present on machine
        Normal   Created         4m51s                  kubelet            Created container logfilesmetricexporter
        Warning  Failed          4m50s                  kubelet            Error: container create failed: time="2022-09-27T07:46:07Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Warning  Failed          4m35s                  kubelet            Error: container create failed: time="2022-09-27T07:46:22Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Warning  Failed          4m24s                  kubelet            Error: container create failed: time="2022-09-27T07:46:33Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Warning  Failed          4m10s                  kubelet            Error: container create failed: time="2022-09-27T07:46:47Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Warning  Failed          3m54s                  kubelet            Error: container create failed: time="2022-09-27T07:47:03Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Warning  Failed          3m40s                  kubelet            Error: container create failed: time="2022-09-27T07:47:17Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Warning  Failed          3m27s                  kubelet            Error: container create failed: time="2022-09-27T07:47:30Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Warning  Failed          3m14s                  kubelet            Error: container create failed: time="2022-09-27T07:47:43Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"
        Normal   Pulled          3m (x11 over 4m51s)    kubelet            Container image "registry.redhat.io/openshift-logging/fluentd-rhel8@sha256:1b8236f3119a6772668f601f8100ac415d403068a58c365efaa3029eba8296f6" already present on machine
        Warning  Failed          3m                     kubelet            (combined from similar events): Error: container create failed: time="2022-09-27T07:47:57Z" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/etc/localtime\" to rootfs at \"/etc/localtime\": mount /etc/localtime:/etc/localtime (via /proc/self/fd/6), flags: 0x5001, data: context=\"system_u:object_r:container_file_t:s0:c309,c770\": not a directory"

              jcantril@redhat.com Jeffrey Cantrill
              gkarager Giriyamma Karagere Ramaswamy (Inactive)
              Giriyamma Karagere Ramaswamy Giriyamma Karagere Ramaswamy (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: