Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-8915

Metering is constantly writing to etcd on vsphere UPI baremetal installations

XMLWordPrintable

    • Quality / Stability / Reliability
    • None
    • None
    • None
    • Moderate
    • None
    • Unspecified
    • None
    • None
    • None
    • None
    • None
    • If docs needed, set a value
    • None
    • None
    • None
    • None
    • None

      Description of problem:
      Follow-up to https://bugzilla.redhat.com/show_bug.cgi?id=1980659.

      When investigating metering's impact on that regression BZ, it was evident from the metering-ansible-operator container logs that requests were either incorrectly proxied, or are being fired off from a non-cached client:

      ```
      {"level":"info","ts":1625863542.6298895,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/network.openshift.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.631801,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/network.operator.openshift.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6333582,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/nfd.openshift.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6348214,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/operators.coreos.com/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6365778,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/ptp.openshift.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.638219,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/samples.operator.openshift.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6398046,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/security.internal.openshift.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6414506,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/snapshot.storage.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.643137,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/sriovnetwork.openshift.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6446972,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/tuned.openshift.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6721356,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/version","Verb":"get","APIPrefix":"","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6744037,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/version/openshift","Verb":"get","APIPrefix":"","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.676316,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis","Verb":"get","APIPrefix":"","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6829047,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/api/v1","Verb":"get","APIPrefix":"","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6912537,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/apiregistration.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.6979203,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/apps/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.7061994,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/events.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.7135413,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/authentication.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.722186,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/authorization.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.730495,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/autoscaling/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.73849,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/batch/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.7466452,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/certificates.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.755377,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/networking.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.7645695,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/rbac.authorization.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.7739985,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/storage.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.7833643,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/admissionregistration.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      {"level":"info","ts":1625863542.7930923,"logger":"proxy","msg":"Skipping cache lookup","resource":{"IsResourceRequest":false,"Path":"/apis/apiextensions.k8s.io/v1","Verb":"get","APIPrefix":"apis","APIGroup":"","APIVersion":"","Namespace":"","Resource":"","Subresource":"","Name":"","Parts":null}}
      ```

      It looks like there's a couple of upstream issues that address those errant log messages:

      Likely worth investigating how often this is reproducible outside of that environment details in that 4.7.20 regression BZ.

      Version-Release number of selected component (if applicable):
      4.7.20

      How reproducible:

      Steps to Reproduce:
      1.
      2.
      3.

      Actual results:

      Expected results:

      Additional info:

              openshift_jira_bot OpenShift Jira Bot
              tflannag@redhat.com Tim Flannagan (Inactive)
              None
              None
              Peter Ruan Peter Ruan
              None
              Red Hat Employee
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: