Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-15576

4.13 LVMS: PVC stuck pending when applying whole LVMS deployment config in a single List

XMLWordPrintable

    • No
    • 2
    • OCP VE Sprint 239, OCP VE Sprint 240
    • 2
    • False
    • Hide

      None

      Show
      None

      Description of problem:
      It seems when passing as a list the pvc did not setup and image-registry goes into a not available mode. This is filed against image-registry component but seems when remove image-registry config and pvc config , re-apply both again it come back up and no longer in this error - but we have pvc that never reconciles on its own. It may need to be moved but leaving open to see.

      ClusterID: bd7abbe9-194b-496c-a6df-2d9b4642733f
      ClusterVersion: Stable at "4.13.0"
      ClusterOperators:
      clusteroperator/image-registry is not available (Available: The deployment does not have available replicas
      NodeCADaemonAvailable: The daemon set node-ca has available replicas
      ImagePrunerAvailable: Pruner CronJob has been created) because Degraded: The deployment does not have available replicas

      Version-Release number of selected component (if applicable):

      How reproducible:

      100%
      
      

      Steps to Reproduce:

      1. Add LVMS operator 4.13
      2. Apply LVMCluster, PersistantVolumeClaim, and Config (Image Registry) in one yaml as a list. (see below)
      3.  oc apply -f List.yaml
      
      oc apply -f List.yaml
      lvmcluster.lvm.topolvm.io/lvms-lvmcluster created
      persistentvolumeclaim/registry-storage created
      Warning: resource configs/cluster is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by oc apply. oc apply should only be used on resources created declaratively by either oc create --save-config or oc apply. The missing annotation will be patched automatically.
      config.imageregistry.operator.openshift.io/cluster configured
      
      
      Actual results:{code:none}
      
      It seems even when you add LVMCluster, and PVC it never sets up PVC properly from the List.
      
       oc get pvc -A
      NAMESPACE                  NAME               STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
      openshift-image-registry   registry-storage   Pending                                                     103s
      
      It should look like:
      oc get pvc -A
      NAMESPACE                  NAME               STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
      openshift-image-registry   registry-storage   Pending                                      lvms-vg1       5s
      
      

      Expected results:

      I would expect it to reconcile

      Additional info:

      apiVersion: v1
      kind: List
      items:

      • apiVersion: lvm.topolvm.io/v1alpha1
        kind: LVMCluster
        metadata:
        name: lvms-lvmcluster
        namespace: openshift-storage
        spec:
        storage:
        nodeSelector:
      • nodeSelectorTerms:
      • matchExpressions:
      • key: kubernetes.io/hostname
        operator: In
        value:
      • sno-0
        deviceClasses:
      • deviceSelector:
        paths:
      • /dev/disk/by-path/pci-0000:31:00.0-nvme-1
        name: vg1
        thinPoolConfig:
        name: thin-pool-1
        overprovisionRatio: 10
        sizePercent: 90
      • apiVersion: v1
        kind: PersistentVolumeClaim
        metadata:
        name: registry-storage
        namespace: openshift-image-registry
        spec:
        accessModes:
      • ReadWriteOnce
        resources:
        requests:
        storage: 120Gi
        storageClassName: lvms-vg1
      • apiVersion: imageregistry.operator.openshift.io/v1
        kind: Config
        metadata:
        name: cluster
        spec:
        defaultRoute: true
        managementState: Managed
        replicas: 1
        rolloutStrategy: Recreate
        storage:
        pvc:
        claim: registry-storage

      oc get pvc -A
      NAMESPACE NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
      openshift-image-registry registry-storage Pending 2m41s

      oc get pods -n openshift-image-registry
      NAME READY STATUS RESTARTS AGE
      cluster-image-registry-operator-5c59d9ffb8-sk95s 1/1 Running 8 123m
      image-registry-858c7b77c7-wcsq9 0/1 Pending 0 2m51s
      node-ca-c9n5j 1/1 Running 8 117m

      oc get deployment -n openshift-image-registry
      NAME READY UP-TO-DATE AVAILABLE AGE
      cluster-image-registry-operator 1/1 1 1 139m
      image-registry 0/1 1 0 18m

      oc get clusterversion
      NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
      version 4.13.0 True False 137m Error while reconciling 4.13.0: the cluster operator image-registry is not available

      oc get co
      NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
      authentication 4.13.0 True False False 112m
      baremetal 4.13.0 True False False 143m
      cloud-controller-manager 4.13.0 True False False 143m
      cloud-credential 4.13.0 True False False 144m
      cluster-autoscaler 4.13.0 True False False 142m
      config-operator 4.13.0 True False False 152m
      console 4.13.0 True False False 19m
      control-plane-machine-set 4.13.0 True False False 143m
      csi-snapshot-controller 4.13.0 True False False 144m
      dns 4.13.0 True False False 98m
      etcd 4.13.0 True False False 149m
      image-registry 4.13.0 False True True 22m Available: The deployment does not have available replicas...
      ingress 4.13.0 True False False 142m
      insights 4.13.0 True False False 143m
      kube-apiserver 4.13.0 True False False 140m
      kube-controller-manager 4.13.0 True False False 140m
      kube-scheduler 4.13.0 True False False 140m
      kube-storage-version-migrator 4.13.0 True False False 145m
      machine-api 4.13.0 True False False 143m
      machine-approver 4.13.0 True False False 143m
      machine-config 4.13.0 True False False 141m
      marketplace 4.13.0 True False False 144m
      monitoring 4.13.0 True False False 19m
      network 4.13.0 True False False 152m
      node-tuning 4.13.0 True False False 143m
      openshift-apiserver 4.13.0 True False False 20m
      openshift-controller-manager 4.13.0 True False False 22m
      openshift-samples 4.13.0 True False False 139m
      operator-lifecycle-manager 4.13.0 True False False 143m
      operator-lifecycle-manager-catalog 4.13.0 True False False 143m
      operator-lifecycle-manager-packageserver 4.13.0 True False False 64m
      service-ca 4.13.0 True False False 151m
      storage 4.13.0 True False False 143m

      Workaround 1:
      If you delete the pvc.yaml, and image-registry.yaml and re-apply this error will go away but it seems valid error from image-registry perspective because there is nothing available to reconcile until redoing

      Workaround 2:
      Add them manually one by one (yaml) and works. This is found trying to automate but think it all should reconcile.

              rh-ee-jmoller Jakob Moeller (Inactive)
              mlammon@redhat.com Mike Lammon
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: