Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-45840

GCP ListRequestsFilterCostOverheadPerMinutePerProject RATE_LIMIT_EXCEEDED

XMLWordPrintable

    • None
    • Installer (PB) Sprint 263, Installer (PB) Sprint 265
    • 2
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      Previously, the installer used GCP Server side filtering to handle queries/filters reducing the number of returned API elements. When number of resources being filtered is large enough, the quota (limit per minute) may be exceeded. The fix removes server side filtering of GCP resources in favor of client side filtering where the installer source code will filter through all return results. The result is avoiding filter quotas for a GCP project.
      Show
      Previously, the installer used GCP Server side filtering to handle queries/filters reducing the number of returned API elements. When number of resources being filtered is large enough, the quota (limit per minute) may be exceeded. The fix removes server side filtering of GCP resources in favor of client side filtering where the installer source code will filter through all return results. The result is avoiding filter quotas for a GCP project.
    • Bug Fix
    • In Progress

      This is a clone of issue OCPBUGS-44193. The following is the description of the original issue:
      —
      In payloads 4.18.0-0.ci-2024-11-01-110334 and 4.18.0-0.nightly-2024-11-01-101707 we observed GCP install failures

       Container test exited with code 3, reason Error
---
ails:
level=error msg=[
level=error msg=  {
level=error msg=    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
level=error msg=    "domain": "googleapis.com",
level=error msg=    "metadatas": {
level=error msg=      "consumer": "projects/711936183532",
level=error msg=      "quota_limit": "ListRequestsFilterCostOverheadPerMinutePerProject",
level=error msg=      "quota_limit_value": "75",
level=error msg=      "quota_location": "global",
level=error msg=      "quota_metric": "compute.googleapis.com/filtered_list_cost_overhead",
level=error msg=      "service": "compute.googleapis.com"
level=error msg=    },
level=error msg=    "reason": "RATE_LIMIT_EXCEEDED"
level=error msg=  },
level=error msg=  {
level=error msg=    "@type": "type.googleapis.com/google.rpc.Help",
level=error msg=    "links": [
level=error msg=      {
level=error msg=        "description": "The request exceeds API Quota limit, please see help link for suggestions.",
level=error msg=        "url": "https://cloud.google.com/compute/docs/api/best-practices#client-side-filter"
level=error msg=      }
level=error msg=    ]
level=error msg=  }
level=error msg=]
level=error msg=, rateLimitExceeded

      padillon Noted ListRequestsFilterCostOverheadPerMinutePerProject can not have it's quota limit increased.

      The problem subsided over the weekend presumably with fewer jobs run but has started to appear again. opening to track ongoing issue and potential work arounds.

      This contributes to the following test failures for GCP

      install should succeed: configuration
install should succeed: overall

              rh-ee-bbarbach Brent Barbachem
              openshift-crt-jira-prow OpenShift Prow Bot
              Gaoyun Pei Gaoyun Pei
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: