Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-9933

3.12.13 quay-builder is corrupted due to a missing gRPC module in quay

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • None
    • quay-v3.12.13
    • quay, quay-builder
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • False

      Description of problem

      3.12.13 quay-builder is corrupted due to a missing gRPC module.

      Version-Release number of selected component (if applicable):

      quay.io/redhat-user-workloads/quay-eng-tenant/stable-3-12-v4-19@sha256:02019b95786eb012e275e7c504173a8f0d38038d1c9c1b6658379b44c89627d3
      
      quay.io/redhat-user-workloads/quay-eng-tenant/quay-builder-v3-12@sha256:6cf256b1c472973870517a2dbbf792ea0abf78b56913c745c82641eaf7e635f3
      

      Steps to reproduce:

      1. Enable builder in quay 3.12.13

      FEATURE_BUILD_SUPPORT: true
      FEATURE_GITLAB_BUILD: true
      GITLAB_TRIGGER_CONFIG:
          CLIENT_ID: ......... 
          CLIENT_SECRET:  ........
      GITLAB_TRIGGER_KIND:
          GITLAB_ENDPOINT: https://gitlab.com/
      FEATURE_BITBUCKET_BUILD: true
      BITBUCKET_TRIGGER_CONFIG:
          CONSUMER_KEY: ......... 
          CONSUMER_SECRET: .........
      FEATURE_GITHUB_BUILD: true
      GITHUB_TRIGGER_CONFIG:
        API_ENDPOINT: https://api.github.com/
        CLIENT_ID: .......
        CLIENT_SECRET:  ..........
        GITHUB_ENDPOINT: https://github.com/
      BUILDMAN_HOSTNAME: quayregistry-quay-builder-quay-enterprise.apps.whu419az33.qe.azure.devcluster.openshift.com:443 
      BUILD_MANAGER:
      - ephemeral
      - ALLOWED_WORKER_COUNT: 20 
        ORCHESTRATOR_PREFIX: buildman/production/
        ORCHESTRATOR:
          REDIS_HOST: quayregistry-quay-redis
          REDIS_PASSWORD: ""
          REDIS_SSL: false
          REDIS_SKIP_KEYSPACE_EVENT_SETUP: false
        EXECUTORS:
        - EXECUTOR: kubernetesPodman
          DEBUG: true
          NAME: openshift
          BUILDER_NAMESPACE: virtual-builds 
          SETUP_TIME: 180
          QUAY_USERNAME: '......
          QUAY_PASSWORD: ...... 
          BUILDER_CONTAINER_IMAGE: quay.io/redhat-user-workloads/quay-eng-tenant/quay-builder-v3-12@sha256:6cf256b1c472973870517a2dbbf792ea0abf78b56913c745c82641eaf7e635f3      # Kubernetes resource options
          K8S_API_SERVER: api.whu419az33.qe.azure.devcluster.openshift.com:6443 
          K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build_cluster.crt
          VOLUME_SIZE: 8G
          KUBERNETES_DISTRIBUTION: openshift
          CONTAINER_MEMORY_LIMITS: 1G 
          CONTAINER_CPU_LIMITS: 1000m
          CONTAINER_MEMORY_REQUEST: 1G 
          CONTAINER_CPU_REQUEST: 500m
          NODE_SELECTOR_LABEL_KEY: ""
          NODE_SELECTOR_LABEL_VALUE: ""
          SERVICE_ACCOUNT_NAME: quay-builder 
          SERVICE_ACCOUNT_TOKEN: .............
      USERFILES_LOCATION: default
      USERFILES_PATH: userfiles 

      2. Trigger a build process

      Actual results:

      Builder can't work, got below errors in quay pod log.

      builder stderr | Traceback (most recent call last):
      builder stderr |   File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
      builder stderr |     return _run_code(code, main_globals, None,
      builder stderr |   File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
      builder stderr |     exec(code, run_globals)
      builder stderr |   File "/quay-registry/buildman/builder.py", line 19, in <module>
      builder stderr |     from buildman.manager.ephemeral import EphemeralBuilderManager
      builder stderr |   File "/quay-registry/buildman/manager/ephemeral.py", line 29, in <module>
      builder stderr |     from buildman.manager.executor import (
      builder stderr |   File "/quay-registry/buildman/manager/executor.py", line 25, in <module>
      builder stderr |     from buildman.server import SECURE_GRPC_SERVER_PORT
      builder stderr |   File "/quay-registry/buildman/server.py", line 7, in <module>
      builder stderr |     import grpc
      builder stderr | ModuleNotFoundError: No module named 'grpc'
      

      Additional information

      Quay 3.12.13 builder worked fine in the previous build.

      quay.io/redhat-user-workloads/quay-eng-tenant/stable-3-12-v4-19@sha256:eea919e64c02e8bccea7d13b959e6f8f1a3acffc2ed742b651cf1081da86757b
      
      quay.io/redhat-user-workloads/quay-eng-tenant/quay-builder-v3-12@sha256:97ff8715ac8d47660eba7342fefe6b16aed3c54544fd5c557d65b611fea2417b
      

              marckok Marcus Kok
              rhwhu Weihua Hu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: