Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-253

kubernetes-nmstate-operator fails to install with error "no channel heads (entries not replaced by another entry) found in channel"

XMLWordPrintable

    • None
    • 1
    • OCP VE Sprint 224
    • 1
    • False
    • Hide

      None

      Show
      None

      +++ This bug was initially created as a clone of Bug #2117223 +++
      +++ This bug was initially created as a clone of Bug #2106935 +++

      Description of problem:

      The nmstate operator fails to install with error

      "message": "no channel heads (entries not replaced by another entry) found in channel \"stable\" of package \"kubernetes-nmstate-operator\"",

      Version-Release number of selected component (if applicable):

      On a newly installed OCP 4.10.15 bare metal cluster

      How reproducible:

      Always. Deleting and reinstalling the operator does not correct the issue.

      Steps to Reproduce:
      1.Create the subscription and wait

      Here is the customer's subscription

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
      annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"operators.coreos.com/v1alpha1","kind":"Subscription","metadata":{"annotations":{},"labels":

      {"app.kubernet es.io/instance":"nerc-ocp-infra-cluster-scope","nerc.mghpcc.org/kustomized":"true"}

      ,"name":"kubernetes-nmstate-operator"
      ,"namespace":"openshift-nmstate"},"spec":{"channel":"stable","installPlanApproval":"Automatic","name":"kubernetes-nmstat
      e-operator","source":"redhat-operators","sourceNamespace":"openshift-marketplace"}}
      creationTimestamp: "2022-06-27T23:02:49Z"
      generation: 3
      labels:
      app.kubernetes.io/instance: nerc-ocp-infra-cluster-scope
      nerc.mghpcc.org/kustomized: "true"
      operators.coreos.com/kubernetes-nmstate-operator.openshift-nmstate: ""
      name: kubernetes-nmstate-operator
      namespace: openshift-nmstate
      resourceVersion: "61147251"
      uid: f6920e77-4a12-40dd-8410-e7a9914aa4e1
      spec:
      channel: stable
      installPlanApproval: Automatic
      name: kubernetes-nmstate-operator
      source: redhat-operators
      sourceNamespace: openshift-marketplace
      startingCSV: kubernetes-nmstate-operator.4.10.0-202205311508
      status:
      catalogHealth:

      catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: certified-operators
      namespace: openshift-marketplace
      resourceVersion: "48033282"
      uid: f619ca87-386c-4d16-a667-bb81c48d1801
      healthy: true
      lastUpdated: "2022-06-27T23:02:51Z"
      catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: community-operators
      namespace: openshift-marketplace
      resourceVersion: "48034491"
      uid: d681196c-a6da-436a-a277-6d145b40672c
      healthy: true
      lastUpdated: "2022-06-27T23:02:51Z"
      catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: redhat-marketplace
      namespace: openshift-marketplace
      resourceVersion: "48024056"
      uid: 8f344b25-8b5a-4e04-8f9a-20a16a91021a
      healthy: true
      lastUpdated: "2022-06-27T23:02:51Z"
      catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: redhat-operators
      namespace: openshift-marketplace
      resourceVersion: "48034492"
      uid: eaba30bc-7e51-49c2-b084-4122e9c717be
      healthy: true
      lastUpdated: "2022-06-27T23:02:51Z"
      conditions:
      lastTransitionTime: "2022-06-27T23:02:51Z"
      message: all available catalogsources are healthy
      reason: AllCatalogSourcesHealthy
      status: "False"
      type: CatalogSourcesUnhealthy
      message: no channel heads (entries not replaced by another entry) found in channel
      "stable" of package "kubernetes-nmstate-operator"
      reason: ErrorPreventedResolution
      status: "True"
      type: ResolutionFailed
      currentCSV: kubernetes-nmstate-operator.4.10.0-202205311508
      installPlanGeneration: 1
      installPlanRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: InstallPlan
      name: install-h4dbm
      namespace: openshift-nmstate
      resourceVersion: "48044077"
      uid: 77dee7cc-5629-43a3-a0cb-99324ab8b70d
      installedCSV: kubernetes-nmstate-operator.4.10.0-202205311508
      installplan:
      apiVersion: operators.coreos.com/v1alpha1
      kind: InstallPlan
      name: install-h4dbm
      uuid: 77dee7cc-5629-43a3-a0cb-99324ab8b70d
      lastUpdated: "2022-07-01T18:10:24Z"
      state: AtLatestKnown
      Actual results:

      type: CatalogSourcesUnhealthy

      message: no channel heads (entries not replaced by another entry) found in channel
      "stable" of package "kubernetes-nmstate-operator"
      reason: ErrorPreventedResolution
      status: "True"
      type: ResolutionFailed
      Expected results:

      The operator installs correctly

      Additional info:

      — Additional comment from dguthrie@redhat.com on 2022-07-13 22:07:31 UTC —

      Description of problem:

      The nmstate operator fails to install with error

      "message": "no channel heads (entries not replaced by another entry) found in channel \"stable\" of package \"kubernetes-nmstate-operator\"",

      Version-Release number of selected component (if applicable):

      On a newly installed OCP 4.10.15 bare metal cluster

      How reproducible:

      Always. Deleting and reinstalling the operator does not correct the issue.

      Steps to Reproduce:
      1.Create the subscription and wait

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
      annotations:
      name: kubernetes-nmstate-operator
      namespace: openshift-nmstate
      spec:
      channel: stable
      installPlanApproval: Automatic
      name: kubernetes-nmstate-operator
      source: redhat-operators
      sourceNamespace: openshift-marketplace
      startingCSV: kubernetes-nmstate-operator.4.10.0-202205311508

      Actual results:

      type: CatalogSourcesUnhealthy

      message: no channel heads (entries not replaced by another entry) found in channel
      "stable" of package "kubernetes-nmstate-operator"
      reason: ErrorPreventedResolution
      status: "True"
      type: ResolutionFailed
      Expected results:

      The operator installs correctly

      Additional info:

      — Additional comment from dguthrie@redhat.com on 2022-07-20 22:38:21 UTC —

      Confusingly, the only issue the appears to happening in the operation of the cluster is that argocd shows degraded resource. It's possible this is masking more operationally significant problems, but we haven't found any other networking issues on the cluster since some version of the nmstate operator did install. We are hoping to get a response before any other problems appear.

      — Additional comment from cstabler@redhat.com on 2022-07-25 08:05:49 UTC —

      Hello dguthrie@redhat.com,
      could you provide the must-gathers so we can have to the other resources?

      — Additional comment from dguthrie@redhat.com on 2022-07-25 17:49:33 UTC —

      Hello @cstabler@redhat.com

      Here is a link to the must-gather in support hub. It's much too large to upload here.

      https://access.redhat.com/support/cases/#/case/03257697/discussion?attachmentId=a096R00002orxBhQAI

      — Additional comment from roarora@redhat.com on 2022-08-01 09:23:29 UTC —

      Another affected customer case.

      Subscription :
      -----------
      spec:
      channel: stable
      installPlanApproval: Automatic
      name: kubernetes-nmstate-operator
      source: redhat-operators
      sourceNamespace: openshift-marketplace
      startingCSV: kubernetes-nmstate-operator.4.10.0-202207140916
      status:
      catalogHealth:

      catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: certified-operators
      namespace: openshift-marketplace
      resourceVersion: "114039328"
      uid: 8fd4aa80-1fce-4cf9-b19a-18c5be55c8d4
      healthy: true
      lastUpdated: "2022-07-29T07:01:48Z"
      catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: community-operators
      namespace: openshift-marketplace
      resourceVersion: "114046184"
      uid: 90ef809d-ee18-498b-b95f-3d5633b535ea
      healthy: true
      lastUpdated: "2022-07-29T07:01:48Z"
      catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: redhat-marketplace
      namespace: openshift-marketplace
      resourceVersion: "114034591"
      uid: 9380b26a-bf1c-4ea1-9c97-4d152a75d287
      healthy: true
      lastUpdated: "2022-07-29T07:01:48Z"
      catalogSourceRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      name: redhat-operators
      namespace: openshift-marketplace
      resourceVersion: "114036727"
      uid: 85323cff-29cc-4ca0-aed7-589ca240d433
      healthy: true
      lastUpdated: "2022-07-29T07:01:48Z"
      conditions:
      lastTransitionTime: "2022-07-29T07:01:48Z"
      message: all available catalogsources are healthy
      reason: AllCatalogSourcesHealthy
      status: "False"
      type: CatalogSourcesUnhealthy
      message: no channel heads (entries not replaced by another entry) found in channel
      "stable" of package "kubernetes-nmstate-operator"
      reason: ErrorPreventedResolution
      status: "True"
      type: ResolutionFailedSubscriotion
      currentCSV: kubernetes-nmstate-operator.4.10.0-202207140916
      installPlanGeneration: 2
      installPlanRef:
      apiVersion: operators.coreos.com/v1alpha1
      kind: InstallPlan
      name: install-fvqgq
      namespace: openshift-nmstate
      resourceVersion: "114048697"
      uid: 833567a8-1e36-48ac-8618-8eab1ea956d9
      installedCSV: kubernetes-nmstate-operator.4.10.0-202207140916
      installplan:
      apiVersion: operators.coreos.com/v1alpha1
      kind: InstallPlan
      name: install-fvqgq
      uuid: 833567a8-1e36-48ac-8618-8eab1ea956d9
      lastUpdated: "2022-07-29T09:15:03Z"
      state: AtLatestKnown
      -----------
      Must gather : https://attachments.access.redhat.com/hydra/rest/cases/03278246/attachments/d86b3d46-fa88-4c6a-908c-4b73ce9f23e2

      — Additional comment from alosingh@redhat.com on 2022-08-03 16:38:22 UTC —

      Hi Team,

      Another customer is facing the same issue on 4.10.24.

      Must gather: https://attachments.access.redhat.com/hydra/rest/cases/03281536/attachments/8f481003-f2a1-424c-85b5-66e028f64b34?usePresignedUrl=true

      — Additional comment from amalykhi@redhat.com on 2022-08-08 12:54:33 UTC —

      Polarion TC https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-47088

      — Additional comment from aos-team-art-private@bot.bugzilla.redhat.com on 2022-08-10 08:05:10 UTC —

      Elliott changed bug status from MODIFIED to ON_QA.
      This bug is expected to ship in the next 4.12 release.

      — Additional comment from amalykhi@redhat.com on 2022-08-10 10:56:43 UTC —

      There are no knmstate builds. The fix will be verified on OCP 4.11

            cstabler@redhat.com Christoph Stäbler
            cstabler@redhat.com Christoph Stäbler
            Aleksandra Malykhin Aleksandra Malykhin
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: