Description of problem:

qemu-img convert with "-n" is much slower than it could, running many fdatasync
during the convert.

Version-Release number of selected component (if applicable):
qemu-img-7.0.0-4.el9.x86_64

How reproducible:
Always

Steps to Reproduce:

1. Create source image

$ qemu-img create -f raw empty.raw 8t
Formatting 'empty.raw', fmt=raw size=8796093022208

2. Create target image

$ qemu-img create -f qcow2 target.qcow2 8t
Formatting 'target.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off
compression_type=zlib size=1099511627776 lazy_refcounts=off refcount_bits=16

3. Convert using -n (keeping the target image)

$ time qemu-img convert -f raw -O qcow2 -t none -T none -n empty.raw target.qcow2

real 1m26.608s
user 0m10.918s
sys 0m11.469s

4. Convert without -n (recreating the target image):

$ time qemu-img convert -f raw -O qcow2 -t none -T none empty.raw target.qcow2

real 0m0.056s
user 0m0.012s
sys 0m0.026s

Actual results:
Converting with -n is slow even when there is no data in the source image.

Expected results:
Faster conversion, paying only for data in the source image.

Additional info:

Running with strace reveals that converting writes lot of data (zero clusters?)
and call fdatasync many times during the convert.

$ strace -f -tt -T -o convert.strace qemu-img convert -f raw -O qcow2 -t none -T none -n empty.raw target.qcow2

This is the typical pattern we see:

$ egrep 'pwrite64(|fdatasync(' convert.strace
...
359864 17:05:15.583796 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 458752 <unfinished ...>
359864 17:05:15.584030 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 462848 <unfinished ...>
359864 17:05:15.584211 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 466944 <unfinished ...>
359864 17:05:15.584383 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 471040 <unfinished ...>
359864 17:05:15.584548 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 475136 <unfinished ...>
359864 17:05:15.584723 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 479232 <unfinished ...>
359864 17:05:15.584891 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 483328 <unfinished ...>
359864 17:05:15.585062 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 487424 <unfinished ...>
359864 17:05:15.585225 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 491520 <unfinished ...>
359864 17:05:15.585391 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 495616 <unfinished ...>
359864 17:05:15.585553 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 499712 <unfinished ...>
359864 17:05:15.585723 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 503808 <unfinished ...>
359864 17:05:15.585890 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 507904 <unfinished ...>
359864 17:05:15.586055 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 512000 <unfinished ...>
359864 17:05:15.586219 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 516096 <unfinished ...>
359864 17:05:15.586382 pwrite64(8, "\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\1"..., 4096, 520192 <unfinished ...>
359864 17:05:15.586544 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 524288 <unfinished ...>
359864 17:05:15.586721 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 528384 <unfinished ...>
359864 17:05:15.586885 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 532480 <unfinished ...>
359864 17:05:15.587016 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 536576 <unfinished ...>
359864 17:05:15.587185 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 540672 <unfinished ...>
359864 17:05:15.587348 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 544768 <unfinished ...>
359864 17:05:15.587510 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 548864 <unfinished ...>
359864 17:05:15.587675 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 552960 <unfinished ...>
359864 17:05:15.587857 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 557056 <unfinished ...>
359864 17:05:15.588033 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 561152 <unfinished ...>
359864 17:05:15.588200 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 565248 <unfinished ...>
359864 17:05:15.588362 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 569344 <unfinished ...>
359864 17:05:15.588529 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 573440 <unfinished ...>
359864 17:05:15.588702 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 577536 <unfinished ...>
359864 17:05:15.588866 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 581632 <unfinished ...>
359864 17:05:15.589032 pwrite64(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 585728 <unfinished ...>
359864 17:05:15.589164 fdatasync(8 <unfinished ...>
359864 17:05:15.591075 pwrite64(8, "\200\0\0\0\0\5\0\0\200\0\0\0\0\6\0\0\200\0\0\0\0\7\0\0\200\0\0\0\0\10\0\0"..., 512, 196608 <unfinished ...>
359864 17:05:15.591237 fdatasync(8 <unfinished ...>
359864 17:05:15.592075 pwrite64(8, "\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\0\0\0\0\0\0\0\0\0\0\0"..., 65536, 131072 <unfinished ...>
359864 17:05:15.592263 fdatasync(8 <unfinished ...>

$ xzgrep pwrite64 convert-pwrite64-fdatasync.strace.xz | wc -l
557056

$ xzgrep fdatasync convert-pwrite64-fdatasync.strace.xz | wc -l
49153

When converting an image, we can skip fdatasync calls during the convert,
and sync once at the end.

Since we know the contents of the source during the conversion, can we allocate
clusters more efficiently (e.g. once for every 1g) instead of incrementally?

Workaround:

Don't use -n. RHV was using it for many cases it was not needed.

In cases when -n is required (target is nbd), use --target-is-zero:

$ time qemu-img convert -f raw -O qcow2 -t none -T none -n --target-is-zero empty.raw target.qcow2

real 0m0.046s
user 0m0.021s
sys 0m0.016s