Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-15779

oc client reporting error about external.metrics.k8s.io/v1beta1 when KEDA is installed but ScaledObject not yet created

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Normal Normal
    • None
    • 4.13
    • Pod Autoscaler
    • Moderate
    • No
    • 3
    • False
    • Hide

      None

      Show
      None
    • Hide
      The oc client reports an error on every command when Custom Metrics Autoscaler is installed and configured, but not in use. The message "couldn't get resource list for external.metrics.k8s.io/v1beta1: Got empty response for: external.metrics.k8s.io/v1beta1" is reported each time an oc command runs. Although harmless, the message may cause confusion. Custom Metrics Autoscaler now properly handles the condition and no errors are returned by oc.
      Show
      The oc client reports an error on every command when Custom Metrics Autoscaler is installed and configured, but not in use. The message "couldn't get resource list for external.metrics.k8s.io/v1beta1: Got empty response for: external.metrics.k8s.io/v1beta1" is reported each time an oc command runs. Although harmless, the message may cause confusion. Custom Metrics Autoscaler now properly handles the condition and no errors are returned by oc.
    • Bug Fix
    • Proposed

      Description of problem:

      After installation of Custom Metrics Autoscaler Operator and configuring KedaController resource, the oc client started to report the below problem.

$ oc get pod
E0704 15:21:31.876262 2904372 memcache.go:255] couldn't get resource list for external.metrics.k8s.io/v1beta1: Got empty response for: external.metrics.k8s.io/v1beta1
NAME                       READY   STATUS             RESTARTS   AGE

$ oc edit deployment curl-ip
E0704 15:21:01.197732 2903319 memcache.go:255] couldn't get resource list for external.metrics.k8s.io/v1beta1: Got empty response for: external.metrics.k8s.io/v1beta1

So every-time oc is invoked, "E0704 15:21:01.197732 2903319 memcache.go:255] couldn't get resource list for external.metrics.k8s.io/v1beta1: Got empty response for: external.metrics.k8s.io/v1beta1" is being reported.

Once a ScaledObject is being created, the error goes away and is also not coming back when the ScaledObject is removed again.

When rolling out Custom Metrics Autoscaler at scale it could be possible that in certain OpenShift Container Platform 4 - Cluster no ScaledObject will be created immediately. Hence the error is disturbing and might impact customer experience.

      Version-Release number of selected component (if applicable):

      OpenShift Container Platform 4.13.4

      How reproducible:

      Always

      Steps to Reproduce:

      1. Install OpenShift Container Platform 4 with Custom Metrics Controller
2. Configure KedaController
3. Run oc command with oc client version 4.13.z

      Actual results:

      $ oc version
Client Version: 4.13.4
Kustomize Version: v4.5.7
Server Version: 4.12.22
Kubernetes Version: v1.26.5+7d22122

$ oc get pod
E0704 15:21:36.351059 2904387 memcache.go:255] couldn't get resource list for external.metrics.k8s.io/v1beta1: Got empty response for: external.metrics.k8s.io/v1beta1
NAME                       READY   STATUS             RESTARTS   AGE

      Expected results:

      $ oc version
Client Version: 4.13.4
Kustomize Version: v4.5.7
Server Version: 4.12.22
Kubernetes Version: v1.26.5+7d22122

$ oc get pod
NAME                       READY   STATUS    RESTARTS   AGE

Even if no initial ScaledObject is being created.

      Additional info:

              joelsmith.redhat Joel Smith
              rhn-support-sreber Simon Reber
              Weinan Liu Weinan Liu
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: