-
Bug
-
Resolution: Done
-
Normal
-
CNV v4.14.0
-
None
-
Quality / Stability / Reliability
-
2
-
False
-
-
False
-
None
-
-
CNV I/U Operators Sprint 258, CNV I/U Operators Sprint 259, CNV I/U Operators Sprint 260
-
None
Description of problem:
alert VirtualMachineCRCErrors was renamed to VMStorageClassWarning
Version-Release number of selected component (if applicable):
cnv v4.14.z
How reproducible:
100%
Steps to Reproduce:
1.deplpy cluster 2.run following "oc get prometheusrule -n openshift-cnv prometheus-k8s-rules-cnv -o json" 3.
Actual results:
{
"alert": "VirtualMachineCRCErrors",
"annotations": {
"description": "{{ $value }} Virtual Machines are in risk of causing CRC errors and major service outages",
"runbook_url": "https://github.com/openshift/runbooks/blob/master/alerts/openshift-virtualization-operator/VirtualMachineCRCErrors.md",
"summary": "When running VMs using ODF storage with 'rbd' mounter or 'rbd.csi.ceph.com provisioner', it will report bad crc/signature errors and cluster performance will be severely degraded if krbd:rxbounce is not set."
},
"expr": "(count(kubevirt_ssp_vm_rbd_block_volume_without_rxbounce \u003e 0) or vector(0)) \u003e 0",
"labels": {
"kubernetes_operator_component": "ssp-operator",
"kubernetes_operator_part_of": "kubevirt",
"operator_health_impact": "none",
"severity": "warning"
}
},
{
"expr": "sum(up{pod=~'ssp-operator.*'}) OR on() vector(0)",
"record": "kubevirt_ssp_operator_up"
},
{
"expr": "sum(up{pod=~'virt-template-validator.*'}) OR on() vector(0)",
"record": "kubevirt_ssp_template_validator_up"
},
{
"expr": "sum(kubevirt_ssp_operator_reconcile_succeeded)",
"record": "kubevirt_ssp_operator_reconcile_succeeded_aggregated"
},
{
"expr": "sum(increase(kubevirt_ssp_template_validator_rejected_total{pod=~'virt-template-validator.*'}[1h])) OR on() vector(0)",
"record": "kubevirt_ssp_template_validator_rejected_increase"
},
{
"expr": "sum(increase(kubevirt_ssp_common_templates_restored_total{pod=~'ssp-operator.*'}[1h])) OR on() vector(0)",
"record": "kubevirt_ssp_common_templates_restored_increase"
}
]
}
]
}
}
Expected results:
alret VMStorageClassWarning should apperas instead
"alert": "VMStorageClassWarning",
"annotations": {
"description": "When running VMs using ODF storage with 'rbd' mounter or 'rbd.csi.ceph.com provisioner', VMs may cause reports of bad crc/signature errors due to certain I/O patterns. Cluster performance can be severely degraded if the number of re-transmissions due to crc errors causes network saturation.",
"runbook_url": "https://github.com/openshift/runbooks/blob/master/alerts/openshift-virtualization-operator/VMStorageClassWarning.md",
"summary": "{{ $value }} Virtual Machines may cause reports of bad crc/signature errors due to certain I/O patterns"
},
"expr": "(count(kubevirt_ssp_vm_rbd_block_volume_without_rxbounce \u003e 0) or vector(0)) \u003e 0",
"labels": {
"kubernetes_operator_component": "ssp-operator",
"kubernetes_operator_part_of": "kubevirt",
"operator_health_impact": "none",
"severity": "warning"
}
},
{
"expr": "sum(up{pod=~'ssp-operator.*'}) OR on() vector(0)",
"record": "kubevirt_ssp_operator_up"
},
{
"expr": "sum(up{pod=~'virt-template-validator.*'}) OR on() vector(0)",
"record": "kubevirt_ssp_template_validator_up"
},
{
"expr": "sum(kubevirt_ssp_operator_reconcile_succeeded)",
"record": "kubevirt_ssp_operator_reconcile_succeeded_aggregated"
},
{
"expr": "sum(increase(kubevirt_ssp_template_validator_rejected_total{pod=~'virt-template-validator.*'}[1h])) OR on() vector(0)",
"record": "kubevirt_ssp_template_validator_rejected_increase"
},
{
"expr": "sum(increase(kubevirt_ssp_common_templates_restored_total{pod=~'ssp-operator.*'}[1h])) OR on() vector(0)",
"record": "kubevirt_ssp_common_templates_restored_increase"
}
]
}
]
}
}
Additional info:
https://issues.redhat.com/browse/CNV-38476
- relates to
-
CNV-38476 Update runbooks VirtualMachineCRCErrors
-
- Closed
-
- mentioned on