Uploaded image for project: 'Container Tools'
  1. Container Tools
  2. RUN-2680

[containers/storage] concurrent map writes in overlay driver

XMLWordPrintable

    • RUN 268

      [2950502970] Upstream Reporter: Paul Holzinger
      Upstream issue status: Closed
      Upstream description:

      <+063ms> # # podman run -d --name myweb -p 5123:80 -v /tmp/CI_s2PP/podman_bats.MFc4Ej/myimage-t8GqI3Z5.tar:/var/www/image.tar:Z -w /var/www quay.io/libpod/testimage:20241011 /bin/busybox-extras httpd -f -p 80
      <+084ms> # Trying to pull quay.io/libpod/testimage:20241011...
               # Getting image source signatures
               # Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
               # Copying blob sha256:33b517cffde0ecb1f424f107b005cdfd614c467b9de2ad334970f800b77a4e70
               # fatal error: concurrent map writes
               #
               # goroutine 66 [running]:
               # internal/runtime/maps.fatal({0x1e0a955?, 0x43?})
               #  /usr/lib/golang/src/runtime/panic.go:1058 +0x18
               # github.com/containers/storage/drivers/overlay.(*Driver).ApplyDiffWithDiffer(0xc000145d40, 0xc000103340?, {0x214cfc0, 0xc000468000?})
               #  /var/tmp/go/src/github.com/containers/podman/vendor/github.com[/containers/storage/drivers/overlay/overlay.go:2242](https://github.com/containers/storage/blob/45e051377c44037fbedecf66ae940ad28d16b557/drivers/overlay/overlay.go#L2242) +0x465
               # github.com/containers/storage.(*layerStore).applyDiffWithDifferNoLock(0xc00040c9c0?, 0x0, {0x214cfc0, 0xc000468000})
               #  /var/tmp/go/src/github.com/containers/podman/vendor/github.com[/containers/storage/layers.go:2567](https://github.com/containers/storage/blob/45e051377c44037fbedecf66ae940ad28d16b557/layers.go#L2567) +0xe4
               # github.com/containers/storage.(*store).PrepareStagedLayer(0x214bd20?, 0x0, {0x214cfc0, 0xc000468000})
               #  /var/tmp/go/src/github.com/containers/podman/vendor/github.com[/containers/storage/store.go:3117](https://github.com/containers/storage/blob/45e051377c44037fbedecf66ae940ad28d16b557/store.go#L3117) +0x53
               # github.com/containers/image/v5/storage.(*storageImageDestination).PutBlobPartial(0xc000503c00, {0x215ac40, 0x3015380}, {0x214bc00, 0xc000012468}, {{0xc00004e6e0, 0x47}, 0x556018, {0x0, 0x0, ...}, ...}, ...)
               #  /var/tmp/go/src/github.com/containers/podman/vendor/github.com[/containers/image/v5/storage/storage_dest.go:417](https://github.com/containers/image/blob/45e051377c44037fbedecf66ae940ad28d16b557/v5/storage/storage_dest.go#L417) +0x332
               # github.com/containers/image/v5/copy.(*imageCopier).copyLayer.func2(0xc0001d9b00, 0x0?, {{0xc00004e6e0, 0x47}, 0x556018, {0x0, 0x0, 0x0}, 0x0, {0xc0002b4150, ...}, ...}, ...)
               #  /var/tmp/go/src/github.com/containers/podman/vendor/github.com[/containers/image/v5/copy/single.go:815](https://github.com/containers/image/blob/45e051377c44037fbedecf66ae940ad28d16b557/v5/copy/single.go#L815) +0x258
               # github.com/containers/image/v5/copy.(*imageCopier).copyLayer(0xc0001d9b00, {0x215ac40, 0x3015380}, {{0xc00004e6e0, 0x47}, 0x556018, {0x0, 0x0, 0x0}, 0x0, ...}, ...)
               #  /var/tmp/go/src/github.com/containers/podman/vendor/github.com[/containers/image/v5/copy/single.go:837](https://github.com/containers/image/blob/45e051377c44037fbedecf66ae940ad28d16b557/v5/copy/single.go#L837) +0x9f6
               # github.com/containers/image/v5/copy.(*imageCopier).copyLayers.func1(0x0, {{0xc00004e6e0, 0x47}, 0x556018, {0x0, 0x0, 0x0}, 0x0, {0xc0002b4150, 0x2b}, ...}, ...)
               #  /var/tmp/go/src/github.com/containers/podman/vendor/github.com[/containers/image/v5/copy/single.go:486](https://github.com/containers/image/blob/45e051377c44037fbedecf66ae940ad28d16b557/v5/copy/single.go#L486) +0x392
               # created by github.com/containers/image/v5/copy.(*imageCopier).copyLayers.func2 in goroutine 1
               #  /var/tmp/go/src/github.com/containers/podman/vendor/github.com[/containers/image/v5/copy/single.go:526](https://github.com/containers/image/blob/45e051377c44037fbedecf66ae940ad28d16b557/v5/copy/single.go#L526) +0x14a
               #

      seen in podman CI https://api.cirrus-ci.com/v1/artifact/task/5618317093765120/html/sys-podman-rawhide-root-host-sqlite.log.html (v5.4 branch)

      AFAICT nothing protects stagingDirsLocks map, it should have mutex I guess.


      Upstream URL: https://github.com/containers/storage/issues/2297

              pholzing@redhat.com Paul Holzinger
              upstream-sync Upstream Sync
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: