Uploaded image for project: 'OpenShift Service Mesh'
  1. OpenShift Service Mesh
  2. OSSM-8608

CNI installation leaves temporary files when the istio cni container is killed during copy binaries

XMLWordPrintable

    • False
    • None
    • False
    • Release Notes
    • Hide
      Previously, if a CNI pod was terminated during the installation phase while copying binaries, the Istio-CNI temporary files could be left on the node's filesystem. If this occurred repeatedly, it could eventually fill up the node's disk space.
      Now, when this situation occurs, any existing temporary files are deleted before the next attempt to copy the CNI binary. This delete operation ensures that only one temporary file per Istio version can exist on the node's filesystem.
      Show
      Previously, if a CNI pod was terminated during the installation phase while copying binaries, the Istio-CNI temporary files could be left on the node's filesystem. If this occurred repeatedly, it could eventually fill up the node's disk space. Now, when this situation occurs, any existing temporary files are deleted before the next attempt to copy the CNI binary. This delete operation ensures that only one temporary file per Istio version can exist on the node's filesystem.

      CNI installation leaves temporary files when the istio cni container is killed during copy binaries.
      If this situation recurs, the number of leftover temporary files will increase and could eventually fill up the disk space.

      More details:
      During the installation of CNI, the `copyBinaries` function is called which calls AtomicCopy which calls AtomicWriteReader where is a creation of a .tmp file https://github.com/istio/istio/blob/master/pkg/file/file.go#L81. At the end of that function, the file is renamed from .tmp to the final name.

      When the container is killed or crashes while executing AtomicWriteReader, the `defer func()` is not invoked, leaving the temporary file undeleted. If this situation recurs, the number of leftover temporary files will increase and could eventually fill up the disk space.

      Space usage increases when a user uses multiple versions of SMCP
      e.g.:

      -rwxr-xr-x. 1 root root  55988224 Nov 18 11:54 v2-3-istio-cni.tmp.432675395
-rwxr-xr-x. 1 root root  83824640 Nov 18 13:24 v2-2-istio-cni.tmp.118105515
-rwxr-xr-x. 1 root root  96030720 Nov 18 15:26 v2-4-istio-cni.tmp.1384614397
-rwxr-xr-x. 1 root root  88485888 Nov 18 15:26 v2-3-istio-cni.tmp.3460554704
-rwxr-xr-x. 1 root root  70041600 Nov 18 15:29 v2-4-istio-cni.tmp.2407875406
-rwxr-xr-x. 1 root root  78065664 Nov 18 17:26 v2-4-istio-cni.tmp.1790603063
-rwxr-xr-x. 1 root root  40251392 Nov 18 19:24 v2-5-istio-cni.tmp.3478282981
-rwxr-xr-x. 1 root root 131276800 Nov 18 19:57 v2-4-istio-cni.tmp.152175550
-rwxr-xr-x. 1 root root  91566080 Nov 18 20:38 v2-5-istio-cni.tmp.2078697749
-rwxr-xr-x. 1 root root 107991040 Nov 18 21:13 v2-4-istio-cni.tmp.4173905327
-rwxr-xr-x. 1 root root  42627072 Nov 18 23:01 v2-3-istio-cni.tmp.3966534084
...

      Upstream issue: https://github.com/istio/istio/issues/54311

              mkralik@redhat.com Matej Kralik
              mkralik@redhat.com Matej Kralik
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: