Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Unresolved
Priority: Normal
Fix Version/s: None
Affects Version/s: 4.11.z
Component/s: Test Framework
Labels:
None

Severity:
Moderate
Release Blocker:
Rejected
Blocked:
False
Blocked Reason:

Hide

None

Show
None

SFDC Cases Counter:
SFDC Cases Links:

Description

Description of problem:

Cluster on OSP16 disconnected upi (see steps for full template/installer information) is not able to upgrade to 4.12.1 successfully because of failure in monitoring cluster operator 
"unable to get secret "sigv4-credentials": secrets "sigv4-credentials" not found"

There is another issue open for this under OCPQE but I believe this is a product issue

Version-Release number of selected component (if applicable):

4.11.25

How reproducible:

100%

Steps to Reproduce:
59_Disconnected UPI on OSP16 with RHCOS & RHEL8.6 FIPS On & OVN & https_proxy & Etcd Encryption on

1. Create OCP cluster with template information above
2. Try upgrading the cluster to 4.12.1-x86_64
3. Error occurs in monitoring cluster operator and cant successfully upgrade

Actual results:

Upgrade fails with monitoring operator not able to progress and showing error

Expected results:

Upgrade is able to have no failing/degraded cluster operators and upgrade is successful to new version

Additional info:

https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/ocp-upgrade/job/upgrade-pipeline/32511/consoleFull

https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/ocp-upgrade/job/upgrade-pipeline/32507/parameters/
59_Disconnected UPI on OSP16 with RHCOS & RHEL8.6 FIPS On & OVN & https_proxy & Etcd Encryption on

01-26 14:11:55.496  oc describe clusteroperators/monitoring:
01-26 14:11:55.497  Name:         monitoring
01-26 14:11:55.497  Namespace:    
01-26 14:11:55.497  Labels:       <none>
01-26 14:11:55.497  Annotations:  include.release.openshift.io/ibm-cloud-managed: true
01-26 14:11:55.497                include.release.openshift.io/self-managed-high-availability: true
01-26 14:11:55.497                include.release.openshift.io/single-node-developer: true
01-26 14:11:55.497  API Version:  config.openshift.io/v1
01-26 14:11:55.497  Kind:         ClusterOperator
01-26 14:11:55.497  Metadata:
01-26 14:11:55.497    Creation Timestamp:  2023-01-26T14:50:05Z
01-26 14:11:55.497    Generation:          1
01-26 14:11:55.497    Managed Fields:
01-26 14:11:55.497      API Version:  config.openshift.io/v1
01-26 14:11:55.497      Fields Type:  FieldsV1
01-26 14:11:55.497      fieldsV1:
01-26 14:11:55.497        f:metadata:
01-26 14:11:55.497          f:annotations:
01-26 14:11:55.497            .:
01-26 14:11:55.497            f:include.release.openshift.io/ibm-cloud-managed:
01-26 14:11:55.497            f:include.release.openshift.io/self-managed-high-availability:
01-26 14:11:55.497            f:include.release.openshift.io/single-node-developer:
01-26 14:11:55.497          f:ownerReferences:
01-26 14:11:55.497            .:
01-26 14:11:55.497            k:{"uid":"47a028d7-5daa-4743-99c4-08d3a20cc9c5"}:
01-26 14:11:55.497        f:spec:
01-26 14:11:55.497      Manager:      Go-http-client
01-26 14:11:55.497      Operation:    Update
01-26 14:11:55.497      Time:         2023-01-26T14:50:05Z
01-26 14:11:55.497      API Version:  config.openshift.io/v1
01-26 14:11:55.497      Fields Type:  FieldsV1
01-26 14:11:55.497      fieldsV1:
01-26 14:11:55.497        f:status:
01-26 14:11:55.497          .:
01-26 14:11:55.497          f:extension:
01-26 14:11:55.497          f:relatedObjects:
01-26 14:11:55.497      Manager:      Go-http-client
01-26 14:11:55.497      Operation:    Update
01-26 14:11:55.497      Subresource:  status
01-26 14:11:55.497      Time:         2023-01-26T14:50:05Z
01-26 14:11:55.497      API Version:  config.openshift.io/v1
01-26 14:11:55.497      Fields Type:  FieldsV1
01-26 14:11:55.497      fieldsV1:
01-26 14:11:55.497        f:status:
01-26 14:11:55.497          f:conditions:
01-26 14:11:55.497          f:versions:
01-26 14:11:55.497      Manager:      operator
01-26 14:11:55.497      Operation:    Update
01-26 14:11:55.497      Subresource:  status
01-26 14:11:55.497      Time:         2023-01-26T18:11:59Z
01-26 14:11:55.497    Owner References:
01-26 14:11:55.497      API Version:     config.openshift.io/v1
01-26 14:11:55.497      Kind:            ClusterVersion
01-26 14:11:55.497      Name:            version
01-26 14:11:55.497      UID:             47a028d7-5daa-4743-99c4-08d3a20cc9c5
01-26 14:11:55.497    Resource Version:  127666
01-26 14:11:55.497    UID:               43992703-8701-41ff-8616-0c59b9a191ff
01-26 14:11:55.497  Spec:
01-26 14:11:55.497  Status:
01-26 14:11:55.497    Conditions:
01-26 14:11:55.497      Last Transition Time:  2023-01-26T18:11:59Z
01-26 14:11:55.497      Reason:                AsExpected
01-26 14:11:55.497      Status:                True
01-26 14:11:55.497      Type:                  Available
01-26 14:11:55.497      Last Transition Time:  2023-01-26T18:11:59Z
01-26 14:11:55.497      Message:               ReconciliationFailed: creating config failed: remote write 0: failed to read SigV4 access-key: unable to get secret "sigv4-credentials": secrets "sigv4-credentials" not found
01-26 14:11:55.497      Reason:                UpdatingPrometheusK8SFailed
01-26 14:11:55.497      Status:                True
01-26 14:11:55.497      Type:                  Degraded
01-26 14:11:55.497      Last Transition Time:  2023-01-26T17:58:57Z
01-26 14:11:55.497      Message:               Rolling out the stack.
01-26 14:11:55.497      Reason:                RollOutInProgress
01-26 14:11:55.497      Status:                True
01-26 14:11:55.497      Type:                  Progressing
01-26 14:11:55.497      Last Transition Time:  2023-01-26T15:39:17Z
01-26 14:11:55.497      Status:                True
01-26 14:11:55.497      Type:                  Upgradeable
01-26 14:11:55.497    Extension:               <nil>
01-26 14:11:55.497    Related Objects:
01-26 14:11:55.497      Group:     
01-26 14:11:55.497      Name:      openshift-monitoring
01-26 14:11:55.497      Resource:  namespaces
01-26 14:11:55.497      Group:     
01-26 14:11:55.497      Name:      openshift-user-workload-monitoring
01-26 14:11:55.497      Resource:  namespaces
01-26 14:11:55.497      Group:     monitoring.coreos.com
01-26 14:11:55.497      Name:      
01-26 14:11:55.497      Resource:  servicemonitors
01-26 14:11:55.497      Group:     monitoring.coreos.com
01-26 14:11:55.497      Name:      
01-26 14:11:55.497      Resource:  podmonitors
01-26 14:11:55.497      Group:     monitoring.coreos.com
01-26 14:11:55.497      Name:      
01-26 14:11:55.497      Resource:  prometheusrules
01-26 14:11:55.497      Group:     monitoring.coreos.com
01-26 14:11:55.497      Name:      
01-26 14:11:55.497      Resource:  alertmanagers
01-26 14:11:55.497      Group:     monitoring.coreos.com
01-26 14:11:55.497      Name:      
01-26 14:11:55.497      Resource:  prometheuses
01-26 14:11:55.497      Group:     monitoring.coreos.com
01-26 14:11:55.497      Name:      
01-26 14:11:55.497      Resource:  thanosrulers
01-26 14:11:55.497      Group:     monitoring.coreos.com
01-26 14:11:55.497      Name:      
01-26 14:11:55.497      Resource:  alertmanagerconfigs
01-26 14:11:55.497    Versions:
01-26 14:11:55.497      Name:     operator
01-26 14:11:55.497      Version:  4.11.25
01-26 14:11:55.497  Events:       <none>
01-26 14:11:55.497

Attachments

Issue Links

is blocked by

MON-2942 impact Monitoring operator fails with error unable to get secret "sigv4-credentials": secrets "sigv4-credentials" not found

Closed

Activity

People

Assignee:: Devan Goodwin

Reporter:: Paige Rubendall

QA Contact:: Junqi Zhao

Votes:: 0 Vote for this issue

Watchers:: 8 Start watching this issue

Dates

Created:: 2023/01/26 8:35 PM

Updated:: 2023/03/02 2:29 PM