[ Arik|mailto:ahadas@redhat.com] 2022-11-27 21:19:38 UTC
When asking for a volume with file system on a block storage for a virtual disk, CDI allocated more than the requested size in order to ensure we'll have enough space for the virtual size within the PV, considering the space that is needed by the file system.

Specifically, if we are about to transfer a disk with virtual size = 100G:
1. MTV asks for a PVC with 100G
2. If the volume is allocated on block storage and contains a file system, it allocates 5.5% more space
3. A PV with 106G is allocated
4. A virtual disk with virtual size = 100G is placed within that file system

That logic generally works fine.
However, when the file system is of type EXT4 there's an additional overhead, besides the above mentioned file system overhead, that is reserved for the root partition. In this case, the extra 5.5% allocation may not be enough to contain the virtual size. CDI executes the following validation at the end of the transfer - it gets the usable space on the file system and compares it with the virtual size of the disk, if the latter is greater it fails the disk transfer.

The way to workaround this issue that we need to document for MTV is to adjust the filesystemOverhead configuration of the storage class in this case, it needs to increase to ~10%
[ RHEL Program Management|mailto:pm-rhel@redhat.com] 2022-11-27 21:19:47 UTC
Flags: devel_ack? qa_ack? pm_ack?
https://bugzilla.redhat.com/show_bug.cgi?id=2148771# [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#] [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#]
Private
  Comment 1 [ Arik|mailto:ahadas@redhat.com] 2022-11-27 21:33:58 UTC RED HAT CONFIDENTIAL

Hi Jochen,
This issue has originally been reported in the migration space [1] and then discussed in the CNV Storage space [2].
Long story short, if we increase the default filesystemOverhead to cover such volumes with EXT4, we'll end up spending more space on other file systems. Since EXT4 is not considered to be a common choice for containers, it doesn't seem to be a good idea so I've opened this bug to document the aforementioned workaround. Does it sound reasonable to consider this a special, uncommon case that should be handled this way?

[1] https://chat.google.com/room/AAAAZOK90b0/zMbqNYycDHQ
[2] https://mail.google.com/mail/u/1/#chat/space/AAAAPkSniUY
Flags: needinfo?(joschrod@redhat.com)
CC: awels@redhat.com, joschrod@redhat.com, thhall@redhat.com
[ Benny Zlotnik|mailto:bzlotnik@redhat.com] 2022-12-04 11:36:15 UTC
CC: bzlotnik@redhat.com
https://bugzilla.redhat.com/show_bug.cgi?id=2148771# [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#] [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#]
Private
  Comment 2 [ Benny Zlotnik|mailto:bzlotnik@redhat.com] 2022-12-04 12:32:53 UTC RED HAT CONFIDENTIAL

(In reply to Arik from comment #1
)> Hi Jochen,
> This issue has originally been reported in the migration space [1] and then
> discussed in the CNV Storage space [2].
> Long story short, if we increase the default filesystemOverhead to cover
> such volumes with EXT4, we'll end up spending more space on other file
> systems. Since EXT4 is not considered to be a common choice for containers,
> it doesn't seem to be a good idea so I've opened this bug to document the
> aforementioned workaround. Does it sound reasonable to consider this a
> special, uncommon case that should be handled this way?
>
> [1] https://chat.google.com/room/AAAAZOK90b0/zMbqNYycDHQ> [2] https://mail.google.com/mail/u/1/#chat/space/AAAAPkSniUYI don't know the numbers, but ceph for example uses ext4 by default[1]

[1] https://github.com/ceph/ceph-csi/blob/ec242d4cc8fd5464117c78567d95ed5bb1e6e8d0/examples/rbd/storageclass.yaml#L81
https://bugzilla.redhat.com/show_bug.cgi?id=2148771# [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#] [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#]
Private Comment 3 [ Jochen Schroder|mailto:joschrod@redhat.com] 2022-12-06 08:47:57 UTC
It absolutely makes sense to correct this - please follow the planned workaround.
Any chance to automate this in Case EXT4 is found?
Flags: needinfo?(joschrod@redhat.com)
https://bugzilla.redhat.com/show_bug.cgi?id=2148771# [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#] [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#]
Private Comment 4 [ Arik|mailto:ahadas@redhat.com] 2022-12-06 12:12:07 UTC
(In reply to Jochen Schroder from comment #3
)> Any chance to automate this in Case EXT4 is found?That probably requires the pods to be privileged, which we'd like to avoid as much as we can
https://bugzilla.redhat.com/show_bug.cgi?id=2148771# [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#] [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#]
Private
  Comment 5 anarnold@redhat.com 2022-12-12 12:10:55 UTC RED HAT CONFIDENTIAL

@ahadas@redhat.com
, just seen this ticket. you assigned it to Anjana and you should have assigned it to me.
Please, could you kindly provide me a quick overview of what the documentation requirements are? I will add it to the list of priorities for the week beginning 19th December, is this acceptable?
Thanks.
CC: anarnold@redhat.com
Flags: needinfo?(ahadas@redhat.com)
anarnold@redhat.com 2022-12-12 12:11:33 UTC
Assignee: asriram@redhat.com → anarnold@redhat.com
https://bugzilla.redhat.com/show_bug.cgi?id=2148771# [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#] [ |https://bugzilla.redhat.com/show_bug.cgi?id=2148771#]
Private
  Comment 6 [ Arik|mailto:ahadas@redhat.com] 2022-12-12 12:18:28 UTC RED HAT CONFIDENTIAL

(In reply to anarnold from comment #5
)> @ahadas@redhat.com , just seen this ticket. you assigned it to Anjana and
> you should have assigned it to me.I think Anjana is the default assignee for MTV bugs, should we ask Robyn to change it then?> Please, could you kindly provide me a quick overview of what the
> documentation requirements are? I will add it to the list of priorities for
> the week beginning 19th December, is this acceptable?Sure, we should add a known issue that migrations may fail due to lack of storage space when using block storage and persistent volumes that are created with EXT4 file system. the issue is that the default overhead that is assumed by CDI doesn't cover the reserved place for the root partition. So a workaround for this is to increase the file system overhead in CDI to be more than 10%.