Uploaded image for project: 'Red Hat OpenBridge'
  1. Red Hat OpenBridge
  2. MGDOBR-1122

@Scheduled method on ConnectorsOIDCClient is executed on only a single pod

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • Internal Service Preview
    • None
    • None
    • None
    • False
    • None
    • False
    • MGDOBR - Sprint 225

      Issue Description: 

      We run Quartz in a clustered setup, meaning that jobs will execute on any pod that is created for a Deployment. This is for redundancy so that jobs continue to execute if a single pod crashes.

      This is causing a problem however with our ConnectorsOIDCClient in that the job to renew the Bearer token to invoke the RHOC API is only ever executed on a single pod. This means that over time, the bearer tokens on pods where the job is not executed expire. Requests are processed by any of the pods, so there is a high possibility of a user having their request processed by a Pod that has an expired bearer token.

      Acceptance Criteria: 

      • Ensure that the job to renew the OIDC token executes on each pod of a deployment

      Additional Information: 

      StackTrace from deployment of connector when invoking the RHOC API:

      org.quartz.core.JobRunShell.run(JobRunShell.java:202)\\n\\tat org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)\\nCaused by: com.openshift.cloud.api.connector.invoker.ApiException: {\\n  \\\"kind\\\": \\\"Error\\\",\\n  \\\"id\\\": \\\"15\\\",\\n  \\\"href\\\": \\\"/api/connector_mgmt/v1/errors/15\\\",\\n  \\\"code\\\": \\\"CONNECTOR-MGMT-15\\\",\\n  \\\"reason\\\": \\\"Bearer token is expired\\\"\\n}\\n\\tat com.openshift.cloud.api.connector.invoker.ApiClient.invokeAPI(ApiClient.java:697)\\n\\tat com.openshift.cloud.api.connector.ConnectorsApi.createConnector(ConnectorsApi.java:88)\\n\\tat com.redhat.service.smartevents.manager.connectors.ConnectorsApiClientImpl.createConnector(ConnectorsApiClientImpl.java:98)\\n\\t... 23 more\\n\",\"errorType\":\"com.redhat.service.smartevents.infra.exceptions.definitions.platform.ConnectorCreationException\",\"errorMessage\":\"Failed to create Connector on Connector Namespace 'ccq71q4p8vi5bv8ndl0g' with HTTP Response Code '401'\ 

              rblake@redhat.com Rob Blake
              rblake@redhat.com Rob Blake
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: