Description of problem:
Pod fails to mount a small (10Mi) Filesystem PVC
output:(size 3072 of data subvolume is too small, minimum 4096 blocks
Version-Release number of selected component (if applicable):
4.15
Steps to Reproduce:
1. Create a PVC: kind: PersistentVolumeClaim apiVersion: v1 metadata: name: pvc-lvms-ten-mi-fs spec: volumeMode: Filesystem accessModes: - ReadWriteOnce resources: requests: storage: 10Mi storageClassName: lvms-vg1 ------ $ oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE pvc-lvms-ten-mi-fs Bound pvc-1bba5d94-aa42-4f8e-a75e-fe16debdbd9a 10Mi RWO lvms-vg1 107s
2. Create a pod: apiVersion: v1 kind: Pod metadata: name: test-pod spec: containers: - command: - sleep - "3600000" image: "quay.io/openshift-cnv/qe-cnv-tests-net-util-container:centos-stream-9" imagePullPolicy: IfNotPresent name: test-pod volumeMounts: - mountPath: /foo name: data-vol volumes: - name: data-vol persistentVolumeClaim: claimName: pvc-lvms-ten-mi-fs
Actual results:
Pod is ContainerCreating $ oc get pods NAME READY STATUS RESTARTS AGE test-pod 0/1 ContainerCreating 0 90s
In pod's events:
$ oc describe pod test-pod .... Warning FailedMount 5s (x5 over 13s) kubelet MountVolume.SetUp failed for volume "pvc-1bba5d94-aa42-4f8e-a75e-fe16debdbd9a" : rpc error: code = Internal desc = mount failed: volume=17c48e86-3a9b-4ef0-83c1-698e0d6fd7b5, error=format of disk "/dev/topolvm/17c48e86-3a9b-4ef0-83c1-698e0d6fd7b5" failed: type:("xfs") target:("/var/lib/kubelet/pods/8b4ae8e7-55c0-45d5-805b-02496d66b782/volumes/kubernetes.io~csi/pvc-1bba5d94-aa42-4f8e-a75e-fe16debdbd9a/mount") options:("nouuid,defaults") errcode:(exit status 1) output:(size 3072 of data subvolume is too small, minimum 4096 blocks
Expected results:
Pod could mount a PVC. If provisioner can't handle a small PVC, it should provison a larger PVC.
Additional info:
Full pod describe:
$ oc describe pod test-pod
Name: test-pod
Namespace: default
Priority: 0
Service Account: default
Node: cnv-qe-infra-23.cnvqe2.lab.eng.rdu2.redhat.com/10.1.156.23
Start Time: Tue, 05 Mar 2024 16:37:22 +0000
Labels: <none>
Annotations: k8s.ovn.org/pod-networks:
{"default":{"ip_addresses":["10.128.0.95/23"],"mac_address":"0a:58:0a:80:00:5f","gateway_ips":["10.128.0.1"],"routes":[{"dest":"10.128.0.0...
Status: Pending
IP:
IPs: <none>
Containers:
test-pod:
Container ID:
Image: quay.io/openshift-cnv/qe-cnv-tests-net-util-container:centos-stream-9
Image ID:
Port: <none>
Host Port: <none>
Command:
sleep
3600000
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/foo from data-vol (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qmcrp (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
data-vol:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: pvc-lvms-ten-mi-fs
ReadOnly: false
kube-api-access-qmcrp:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
ConfigMapName: openshift-service-ca.crt
ConfigMapOptional: <nil>
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m4s default-scheduler Successfully assigned default/test-pod to cnv-qe-infra-23.cnvqe2.lab.eng.rdu2.redhat.com
Warning FailedMount 59s (x8 over 2m4s) kubelet MountVolume.SetUp failed for volume "pvc-1bba5d94-aa42-4f8e-a75e-fe16debdbd9a" : rpc error: code = Internal desc = mount failed: volume=17c48e86-3a9b-4ef0-83c1-698e0d6fd7b5, error=format of disk "/dev/topolvm/17c48e86-3a9b-4ef0-83c1-698e0d6fd7b5" failed: type:("xfs") target:("/var/lib/kubelet/pods/8b4ae8e7-55c0-45d5-805b-02496d66b782/volumes/kubernetes.io~csi/pvc-1bba5d94-aa42-4f8e-a75e-fe16debdbd9a/mount") options:("nouuid,defaults") errcode:(exit status 1) output:(size 3072 of data subvolume is too small, minimum 4096 blocks
Usage: mkfs.xfs
/* blocksize */ [-b size=num]
/* config file */ [-c options=xxx]
/* metadata */ [-m crc=0|1,finobt=0|1,uuid=xxx,rmapbt=0|1,reflink=0|1,
inobtcount=0|1,bigtime=0|1]
/* data subvol */ [-d agcount=n,agsize=n,file,name=xxx,size=num,
(sunit=value,swidth=value|su=num,sw=num|noalign),
sectsize=num
/* force overwrite */ [-f]
/* inode size */ [-i perblock=n|size=num,maxpct=n,attr=0|1|2,
projid32bit=0|1,sparse=0|1,nrext64=0|1]
/* no discard */ [-K]
/* log subvol */ [-l agnum=n,internal,size=num,logdev=xxx,version=n
sunit=value|su=num,sectsize=num,lazy-count=0|1]
/* label */ [-L label (maximum 12 characters)]
/* naming */ [-n size=num,version=2|ci,ftype=0|1]
/* no-op info only */ [-N]
/* prototype file */ [-p fname]
/* quiet */ [-q]
/* realtime subvol */ [-r extsize=num,size=num,rtdev=xxx]
/* sectorsize */ [-s size=num]
/* version */ [-V]
devicename
<devicename> is required unless -d name=xxx is given.
<num> is xxx (bytes), xxxs (sectors), xxxb (fs blocks), xxxk (xxx KiB),
xxxm (xxx MiB), xxxg (xxx GiB), xxxt (xxx TiB) or xxxp (xxx PiB).
<value> is xxx (512 byte blocks).
)
- is cloned by
-
OCPBUGS-32160 [release-4.15] LVMS: FailedMount a small (10Mi) Filesystem PVC
-
- Closed
-
- is depended on by
-
OCPBUGS-32160 [release-4.15] LVMS: FailedMount a small (10Mi) Filesystem PVC
-
- Closed
-
- links to
-
RHBA-2024:129552
LVMS 4.16 Bug Fix and Enhancement update
(4 links to)