Loading...

XML

Word

Printable

Type: Bug
Resolution: Done-Errata
Priority: Major
Fix Version/s: 4.14.0
Affects Version/s: 4.14
Component/s: Monitoring
Labels:

Severity:
Important
Regression:
No
Sprint:
MON Sprint 238
sprint_count:
1
Release Blocker:
Approved
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Release Note Text:
N/A
Release Note Type:
Release Note Not Required
Internal Whiteboard:
Target Version:

4.14.0

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

Description of problem:

When deploying 4.14 spoke, agentclusterinstall is stuck at finalizing stage

clusterverions on spoke report "Unable to apply 4.14.0-0.ci-2023-06-13-083232: the cluster operator monitoring is not available"

Please note: console operator is disabled purposely - it is needed in telco case to reduce platform resource usage

[kni@registry.kni-qe-28 ~]$ oc get clusterversions.config.openshift.io -A
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version             False       True          46m     Unable to apply 4.14.0-0.ci-2023-06-13-083232: the cluster operator monitoring is not available

[kni@registry.kni-qe-28 ~]$ oc get clusterversions.config.openshift.io -n version -o yaml 
apiVersion: v1
items:
- apiVersion: config.openshift.io/v1
  kind: ClusterVersion
  metadata:
    creationTimestamp: "2023-06-13T15:16:32Z"
    generation: 2
    name: version
    resourceVersion: "20061"
    uid: f8fc0c3e-009d-4d86-a05d-2fd0aba59528
  spec:
    capabilities:
      additionalEnabledCapabilities:
      - marketplace
      - NodeTuning
      baselineCapabilitySet: None
    channel: stable-4.14
    clusterID: 5cfc0491-5a23-4383-935b-71e3c793e875
  status:
    availableUpdates: null
    capabilities:
      enabledCapabilities:
      - NodeTuning
      - marketplace
      knownCapabilities:
      - CSISnapshot
      - Console
      - Insights
      - NodeTuning
      - Storage
      - baremetal
      - marketplace
      - openshift-samples
    conditions:
    - lastTransitionTime: "2023-06-13T15:16:33Z"
      message: 'Unable to retrieve available updates: Get "https://api.openshift.com/api/upgrades_info/v1/graph?arch=amd64&channel=stable-4.14&id=5cfc0491-5a23-4383-935b-71e3c793e875&version=4.14.0-0.ci-2023-06-13-083232":
        dial tcp 54.211.39.83:443: connect: network is unreachable'
      reason: RemoteFailed
      status: "False"
      type: RetrievedUpdates
    - lastTransitionTime: "2023-06-13T15:16:33Z"
      message: Capabilities match configured spec
      reason: AsExpected
      status: "False"
      type: ImplicitlyEnabledCapabilities
    - lastTransitionTime: "2023-06-13T15:16:33Z"
      message: Payload loaded version="4.14.0-0.ci-2023-06-13-083232" image="registry.kni-qe-28.ptp.lab.eng.bos.redhat.com:5000/openshift-release-dev/ocp-release@sha256:826bb878c5a1469ee8bb991beebc38a4e25b8f5cef9cdf1931ef99ffe5ffbc80"
        architecture="amd64"
      reason: PayloadLoaded
      status: "True"
      type: ReleaseAccepted
    - lastTransitionTime: "2023-06-13T15:16:33Z"
      status: "False"
      type: Available
    - lastTransitionTime: "2023-06-13T15:41:36Z"
      message: Cluster operator monitoring is not available
      reason: ClusterOperatorNotAvailable
      status: "True"
      type: Failing
    - lastTransitionTime: "2023-06-13T15:16:33Z"
      message: 'Unable to apply 4.14.0-0.ci-2023-06-13-083232: the cluster operator
        monitoring is not available'
      reason: ClusterOperatorNotAvailable
      status: "True"
      type: Progressing
    desired:
      image: registry.kni-qe-28.ptp.lab.eng.bos.redhat.com:5000/openshift-release-dev/ocp-release@sha256:826bb878c5a1469ee8bb991beebc38a4e25b8f5cef9cdf1931ef99ffe5ffbc80
      version: 4.14.0-0.ci-2023-06-13-083232
    history:
    - completionTime: null
      image: registry.kni-qe-28.ptp.lab.eng.bos.redhat.com:5000/openshift-release-dev/ocp-release@sha256:826bb878c5a1469ee8bb991beebc38a4e25b8f5cef9cdf1931ef99ffe5ffbc80
      startedTime: "2023-06-13T15:16:33Z"
      state: Partial
      verified: false
      version: 4.14.0-0.ci-2023-06-13-083232
    observedGeneration: 2
    versionHash: H6tRc6p_ZWU=
kind: List
metadata:
  resourceVersion: ""

[kni@registry.kni-qe-28 ~]$ oc get co -A
NAME                                       VERSION                         AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
authentication                             4.14.0-0.ci-2023-06-13-083232   True        False         False      14m     
cloud-controller-manager                   4.14.0-0.ci-2023-06-13-083232   True        False         False      24m     
cloud-credential                           4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
cluster-autoscaler                         4.14.0-0.ci-2023-06-13-083232   True        False         False      24m     
config-operator                            4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
control-plane-machine-set                  4.14.0-0.ci-2023-06-13-083232   True        False         False      24m     
dns                                        4.14.0-0.ci-2023-06-13-083232   True        False         False      19m     
etcd                                       4.14.0-0.ci-2023-06-13-083232   True        False         False      22m     
image-registry                             4.14.0-0.ci-2023-06-13-083232   True        False         False      14m     
ingress                                    4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
kube-apiserver                             4.14.0-0.ci-2023-06-13-083232   True        False         False      18m     
kube-controller-manager                    4.14.0-0.ci-2023-06-13-083232   True        False         False      19m     
kube-scheduler                             4.14.0-0.ci-2023-06-13-083232   True        False         False      17m     
kube-storage-version-migrator              4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
machine-api                                4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
machine-approver                           4.14.0-0.ci-2023-06-13-083232   True        False         False      24m     
machine-config                             4.14.0-0.ci-2023-06-13-083232   True        False         False      21m     
marketplace                                4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
monitoring                                                                 False       True          True       14m     reconciling Console Plugin failed: creating ConsolePlugin object failed: the server could not find the requested resource (post consoleplugins.console.openshift.io)
network                                    4.14.0-0.ci-2023-06-13-083232   True        False         False      26m     
node-tuning                                4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
openshift-apiserver                        4.14.0-0.ci-2023-06-13-083232   True        False         False      14m     
openshift-controller-manager               4.14.0-0.ci-2023-06-13-083232   True        False         False      18m     
operator-lifecycle-manager                 4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
operator-lifecycle-manager-catalog         4.14.0-0.ci-2023-06-13-083232   True        False         False      25m     
operator-lifecycle-manager-packageserver   4.14.0-0.ci-2023-06-13-083232   True        False         False      19m     
service-ca                                 4.14.0-0.ci-2023-06-13-083232   True        False         False      25m

Version-Release number of selected component (if applicable):
4.14

How reproducible:

100%

Steps to Reproduce:

1. Deploy RAN DU spoke cluster via gitops ZTP approach with multiple base capabilities disabled including Console operator.
   spec:     
     capabilities:       
       additionalEnabledCapabilities:
         - marketplace       
         - NodeTuning       
     baselineCapabilitySet: None     
     channel: stable-4.14 
2. Monitor ocp deployment on spoke.

Actual results:

Deployment fails while finalizing agentclusterinstall.  clusterverions on spoke report "the cluster operator monitoring is not available"

Expected results:

Successful spoke deployment

Additional info:

After manually enabling console in clusterversion, the monitoring operator succeeded and OCP install completed

must-gather logs:
https://drive.google.com/file/d/19zO21jqcVTIkAdGS2DEqQuhg2oGUmuNY/view?usp=sharing
https://drive.google.com/file/d/1PXjZmBdMwHWNwkaXr2wE9tTtBRJWYeKP/view?usp=sharing

links to

openshift/cluster-monitoring-operator#2011: OCPBUGS-14922: skip console-plugin installation if console CO is absent

RHSA-2023:5006 OpenShift Container Platform 4.14.z security update

Assignee:: Sunil Thaha

Reporter:: Bonnie Block

QA Contact:: Tai Gao

Votes:: 0 Vote for this issue

Watchers:: 13 Start watching this issue

Created:: 2023/06/13 4:42 PM

Updated:: 2023/10/31 1:39 PM

Resolved:: 2023/10/31 1:39 PM

Details

Description

Attachments

Issue Links

Activity

People

Dates