Uploaded image for project: 'OpenShift Node'
  1. OpenShift Node
  2. OCPNODE-1621

Log Linking via CRIO

XMLWordPrintable

    • Icon: Epic Epic
    • Resolution: Done
    • Icon: Critical Critical
    • openshift-4.14
    • None
    • None
    • None
    • Log Linking
    • False
    • None
    • False
    • Not Selected
    • To Do
    • OCPSTRAT-100 - Log linking via CRIO
    • OCPSTRAT-100Log linking via CRIO
    • 100
    • 100% 100%

      Epic Goal

      • Support making containers logs in a pod available inside an emptyDir so they could be read/forwarded by a logging sidecar.

      Why is this important?

      • Customers want to be able to access container logs easily inside the pods so they could read them and log them to their sinks using logging side car containers. 

      Scenarios

      1. As an OpenShift cluster admin, I want my user to be able to forward logs using a logging side care without having to modify their application entrypoint.

      Design

      1. The admin labels namespaces that allow pods to request to hard link their logs through annotations (This is like a downward API for accessing logs). 
      2. A pod adds an annotation like "io.crio.link-logs": "true" or "name-of-emptyDir". 
      3. An admission controller checks if the pod's namespace allows that; if not, it removes the annotation, else allows it through.
      4. Another admission controller ensures that there is an empty Dir volume in a pod with the desired name for logging. 

          (This part could also be modified to instead require users to specify the volume. I see their current code references requiring pod uid downward reference.)

             5. CRI-O processes the annotation and hard links the logs to either pre-configured or specified emptyDir volume. (We are debating whether to have additional knob in CRI-O to put this into allowed_annotations or not.) CRI-O sets the right SELinux label for the log file.

            6. When CRI-O gets a log rotation request from the kubelet, it makes sure that the hard link is updated as well. This lets us get rid of the service that is 

      using inotify watch to do this work. 

      CRI-O sets the right SELinux label for the log file.

       

      Alternative CRI-O steps:

            5.  CRI-O processes the annotation on pod run and bind mounts the pod log directory to either pre-configured or specified emptyDir volume.
                 CRI-O sets the right SELinux label for the log file.

            6. When CRI-O gets a log rotation request from the kubelet, it only needs to set the right SELinux label for the log file. 

       

      Note: The admission controller work is out of scope for this epic. This epic is mainly concerned with CRI-O linking the logs once it sees the annotation. 

      Acceptance Criteria

      • CI - MUST be running successfully with tests automated
      • Release Technical Enablement - Provide necessary release enablement details and documents.

      Dependencies (internal and external)

      1. ...

      Previous Work (Optional):

      1.  

      Open questions::

      1. Should we require crio allowed_annotations for this?

      Done Checklist

      • CI - CI is running, tests are automated and merged.
      • Release Enablement <link to Feature Enablement Presentation>
      • DEV - Upstream code and tests merged: <link to meaningful PR or GitHub Issue>
      • DEV - Upstream documentation merged: <link to meaningful PR or GitHub Issue>
      • DEV - Downstream build attached to advisory: <link to errata>
      • QE - Test plans in Polarion: <link or reference to Polarion>
      • QE - Automated tests merged: <link or reference to automated tests>
      • DOC - Downstream documentation merged: <link to meaningful PR>

            pehunt@redhat.com Peter Hunt
            gausingh@redhat.com Gaurav Singh
            Min Li Min Li
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: