Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-5430

Builds are set to false yet Quay's buildlogsarchiver throws exception

XMLWordPrintable

    • False
    • None
    • False
    • Quay Enterprise
    • 0

      • following exception is thrown every 2 mins, crowding quay logs 

      buildlogsarchiver stdout | 2023-04-24 19:24:43,366 [53] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-04-24 19:25:13 UTC)" (scheduled at 2023-04-24 19:24:43.366079+00:00)
      buildlogsarchiver stdout | 2023-04-24 19:24:43,384 [53] [ERROR] [workers.worker] Operation raised exception
      buildlogsarchiver stdout | Traceback (most recent call last):
      buildlogsarchiver stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func
      buildlogsarchiver stdout | return operation_func()
      buildlogsarchiver stdout | File "/quay-registry/workers/buildlogsarchiver/buildlogsarchiver.py", line 57, in _archive_redis_buildlogs
      buildlogsarchiver stdout | log_archive.store_file(
      buildlogsarchiver stdout | File "/quay-registry/data/userfiles.py", line 114, in store_file
      buildlogsarchiver stdout | self._storage.stream_write(
      buildlogsarchiver stdout | File "/quay-registry/storage/distributedstorage.py", line 24, in wrapper
      buildlogsarchiver stdout | storage = self._storages[random.sample(locations, 1)[0]]
      buildlogsarchiver stdout | KeyError: 'local_us'
      buildlogsarchiver stdout | 2023-04-24 19:24:43,384 [53] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2023-04-24 19:25:13 UTC)" executed successfully

      • Isn't ArchiveBuildLogsWorker called if builds are turned on in quay? Customer has FEATURE_BUILD_SUPPORT = false
      • exception is seen at line 24 for keyerror. What key was distributed storage not able to retrieve? Is it the case that storage name was not properly set in quay config yaml and it is throwing exception that storage name local_us not found?
      • Is the problem here ?
      •  Again From schema.py I can check
        "LOG_ARCHIVE_LOCATION": {
                    "type": "string",
                    "description": "If builds are enabled, the storage engine in which to place the "
                    + "archived build logs.",
                    "x-example": "s3_us_east",
                },
                
        "LOG_ARCHIVE_PATH": {
                    "type": "string",
                    "description": "If builds are enabled, the path in storage in which to place the "
                    + "archived build logs.",
                    "x-example": "archives/buildlogs",
                },
        

      Kindly help. Thanks

            Unassigned Unassigned
            rhn-support-sbhavsar Sayali Bhavsar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: