Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-44193

GCP ListRequestsFilterCostOverheadPerMinutePerProject RATE_LIMIT_EXCEEDED

XMLWordPrintable

    • None
    • Installer Sprint 262, Installer (PB) Sprint 263, Installer (PB) Sprint 265
    • 3
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, during a {gcp-first} installation, when the installer was creating filters with large numbers of returned data, for example for subnets, it exceeded the quota for the maximum number times that a resource can be filtered in a specific period. With this release, all relevant filtering is moved to the client so that the filter quotas are not exceeded and the issue is resolved. (link:https://issues.redhat.com/browse/OCPBUGS-44193[*OCPBUGS-44193*])
      Show
      * Previously, during a {gcp-first} installation, when the installer was creating filters with large numbers of returned data, for example for subnets, it exceeded the quota for the maximum number times that a resource can be filtered in a specific period. With this release, all relevant filtering is moved to the client so that the filter quotas are not exceeded and the issue is resolved. (link: https://issues.redhat.com/browse/OCPBUGS-44193 [* OCPBUGS-44193 *])
    • Bug Fix
    • Proposed

      In payloads 4.18.0-0.ci-2024-11-01-110334 and 4.18.0-0.nightly-2024-11-01-101707 we observed GCP install failures

       Container test exited with code 3, reason Error
      ---
      ails:
      level=error msg=[
      level=error msg=  {
      level=error msg=    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
      level=error msg=    "domain": "googleapis.com",
      level=error msg=    "metadatas": {
      level=error msg=      "consumer": "projects/711936183532",
      level=error msg=      "quota_limit": "ListRequestsFilterCostOverheadPerMinutePerProject",
      level=error msg=      "quota_limit_value": "75",
      level=error msg=      "quota_location": "global",
      level=error msg=      "quota_metric": "compute.googleapis.com/filtered_list_cost_overhead",
      level=error msg=      "service": "compute.googleapis.com"
      level=error msg=    },
      level=error msg=    "reason": "RATE_LIMIT_EXCEEDED"
      level=error msg=  },
      level=error msg=  {
      level=error msg=    "@type": "type.googleapis.com/google.rpc.Help",
      level=error msg=    "links": [
      level=error msg=      {
      level=error msg=        "description": "The request exceeds API Quota limit, please see help link for suggestions.",
      level=error msg=        "url": "https://cloud.google.com/compute/docs/api/best-practices#client-side-filter"
      level=error msg=      }
      level=error msg=    ]
      level=error msg=  }
      level=error msg=]
      level=error msg=, rateLimitExceeded 
      

      padillon Noted ListRequestsFilterCostOverheadPerMinutePerProject can not have it's quota limit increased.

      The problem subsided over the weekend presumably with fewer jobs run but has started to appear again. opening to track ongoing issue and potential work arounds.

      This contributes to the following test failures for GCP

      install should succeed: configuration
      install should succeed: overall
      

              rh-ee-bbarbach Brent Barbachem
              rh-ee-fbabcock Forrest Babcock
              Jianli Wei Jianli Wei
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated: