__ __ / \ / \ ______ _ _ __ __ __ / /\ / /\ \ / __ \ | | | | / \ \ \ / / / / / / \ \ | | | | | | | | / /\ \ \ / \ \ \ \ / / | |__| | | |__| | / ____ \ | | \ \/ \ \/ / \_ ___/ \____/ /_/ \_\ |_| \__/ \__/ \ \__ \___\ by Red Hat Build, Store, and Distribute your Containers Startup timestamp: Fri Jul 29 09:00:27 UTC 2022 Running all default registry services without migration Running init script '/quay-registry/conf/init/certs_create.sh' Generating a RSA private key ........++++ ...............................................................++++ writing new private key to 'mitm-key.pem' ----- Running init script '/quay-registry/conf/init/certs_install.sh' Installing extra certificates found in /quay-registry/conf/stack/extra_ca_certs directory Running init script '/quay-registry/conf/init/copy_config_files.sh' Running init script '/quay-registry/conf/init/d_validate_config_bundle.sh' Validating Configuration time="2022-07-29T09:00:28Z" level=debug msg="Validating AccessSettings" time="2022-07-29T09:00:28Z" level=debug msg="Validating ActionLogArchiving" time="2022-07-29T09:00:28Z" level=debug msg="Validating AppTokenAuthentication" time="2022-07-29T09:00:28Z" level=debug msg="Validating BitbucketBuildTrigger" time="2022-07-29T09:00:28Z" level=debug msg="Validating BuildManager" time="2022-07-29T09:00:28Z" level=debug msg="Validating Database" time="2022-07-29T09:00:28Z" level=debug msg="Scheme: mysql+pymysql" time="2022-07-29T09:00:28Z" level=debug msg="Host: mpcrdcmysqldb01.prodpcr.rbi.org.in:3306" time="2022-07-29T09:00:28Z" level=debug msg="Db: quay" time="2022-07-29T09:00:28Z" level=debug msg="Params: " time="2022-07-29T09:00:28Z" level=debug msg="CA Cert provided" time="2022-07-29T09:00:28Z" level=debug msg="Database%!(EXTRA string=Could not connect to database. Error: Could not find database.pem in config bundle, []string=[DB_URI])" time="2022-07-29T09:00:28Z" level=debug msg="Validating DistributedStorage" time="2022-07-29T09:00:28Z" level=debug msg="DistributedStorage%!(EXTRA string=Could not connect to storage default. Error: A header you provided implies functionality that is not implemented., []string=[DISTRIBUTED_STORAGE_CONFIG])" time="2022-07-29T09:00:28Z" level=debug msg="Validating ElasticSearch" time="2022-07-29T09:00:28Z" level=debug msg="Validating Email" time="2022-07-29T09:00:28Z" level=debug msg="Validating GitHubBuildTrigger" time="2022-07-29T09:00:28Z" level=debug msg="Validating GitHubLogin" time="2022-07-29T09:00:28Z" level=debug msg="Validating GitLabBuildTrigger" time="2022-07-29T09:00:28Z" level=debug msg="Validating GoogleLogin" time="2022-07-29T09:00:28Z" level=debug msg="Validating HostSettings" time="2022-07-29T09:00:28Z" level=debug msg="Validating JWTAuthentication" time="2022-07-29T09:00:28Z" level=debug msg="Validating LDAP" time="2022-07-29T09:00:28Z" level=debug msg="Validating OIDC" time="2022-07-29T09:00:28Z" level=debug msg="Validating QuayDocumentation" time="2022-07-29T09:00:28Z" level=debug msg="Validating Redis" time="2022-07-29T09:00:28Z" level=debug msg="Address: quay-registry-quay-redis:6379" time="2022-07-29T09:00:28Z" level=debug msg="Username: " time="2022-07-29T09:00:28Z" level=debug msg="Password Len: 0" time="2022-07-29T09:00:28Z" level=debug msg="Ssl: " time="2022-07-29T09:00:28Z" level=debug msg="Address: quay-registry-quay-redis:6379" time="2022-07-29T09:00:28Z" level=debug msg="Username: " time="2022-07-29T09:00:28Z" level=debug msg="Password Len: 0" time="2022-07-29T09:00:28Z" level=debug msg="Ssl: " time="2022-07-29T09:00:28Z" level=debug msg="Validating RepoMirror" time="2022-07-29T09:00:28Z" level=debug msg="Validating SecurityScanner" time="2022-07-29T09:00:28Z" level=debug msg="Validating TeamSyncing" time="2022-07-29T09:00:28Z" level=debug msg="Validating TimeMachine" time="2022-07-29T09:00:28Z" level=debug msg="Validating UserVisibleSettings" +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | Field Group | Error | Status | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | AccessSettings | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | ActionLogArchiving | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | AppTokenAuthentication | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | BitbucketBuildTrigger | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | BuildManager | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | Database | Could not connect to database. Error: Could not find database.pem in config bundle | 🔴 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | DistributedStorage | Could not connect to storage default. Error: A header you provided implies functionality that is not implemented. | 🔴 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | ElasticSearch | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | Email | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | GitHubBuildTrigger | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | GitHubLogin | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | GitLabBuildTrigger | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | GoogleLogin | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | HostSettings | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | JWTAuthentication | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | LDAP | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | OIDC | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | QuayDocumentation | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | Redis | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | RepoMirror | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | SecurityScanner | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | TeamSyncing | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | TimeMachine | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ | UserVisibleSettings | - | 🟢 | +------------------------+-------------------------------------------------------------------------------------------------------------------+--------+ Running init script '/quay-registry/conf/init/nginx_conf_create.sh' Running init script '/quay-registry/conf/init/supervisord_conf_create.sh' Running init script '/quay-registry/conf/init/zz_boot.sh' /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) self._legacy_model = V2SecurityScanner(app, instance_keys, storage) Failed to validate security scanner V2 configuration /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) self._legacy_model = NoopV2SecurityScanner() Generated new service key _qsAOUj_X55KJhZ4Cw--Hxq-be1q7_pCyvSl3ybw3po 2022-07-29 09:00:31,052 INFO RPC interface 'supervisor' initialized 2022-07-29 09:00:31,052 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2022-07-29 09:00:31,052 INFO supervisord started with pid 7 2022-07-29 09:00:32,056 INFO spawned: 'stdout' with pid 62 2022-07-29 09:00:32,059 INFO spawned: 'blobuploadcleanupworker' with pid 63 2022-07-29 09:00:32,062 INFO spawned: 'builder' with pid 64 2022-07-29 09:00:32,064 INFO spawned: 'buildlogsarchiver' with pid 65 2022-07-29 09:00:32,067 INFO spawned: 'chunkcleanupworker' with pid 66 2022-07-29 09:00:32,069 INFO spawned: 'dnsmasq' with pid 67 2022-07-29 09:00:32,072 INFO spawned: 'expiredappspecifictokenworker' with pid 68 2022-07-29 09:00:32,075 INFO spawned: 'exportactionlogsworker' with pid 69 2022-07-29 09:00:32,078 INFO spawned: 'gcworker' with pid 70 2022-07-29 09:00:32,081 INFO spawned: 'globalpromstats' with pid 71 2022-07-29 09:00:32,084 INFO spawned: 'gunicorn-registry' with pid 72 2022-07-29 09:00:32,105 INFO spawned: 'gunicorn-secscan' with pid 73 2022-07-29 09:00:32,107 INFO spawned: 'gunicorn-web' with pid 74 2022-07-29 09:00:32,110 INFO spawned: 'jwtproxy' with pid 75 2022-07-29 09:00:32,112 INFO spawned: 'logrotateworker' with pid 76 2022-07-29 09:00:32,115 INFO spawned: 'manifestbackfillworker' with pid 77 2022-07-29 09:00:32,205 INFO spawned: 'memcache' with pid 81 2022-07-29 09:00:32,208 INFO spawned: 'namespacegcworker' with pid 83 2022-07-29 09:00:32,211 INFO spawned: 'nginx' with pid 84 2022-07-29 09:00:32,214 INFO spawned: 'notificationworker' with pid 90 2022-07-29 09:00:32,216 INFO spawned: 'pushgateway' with pid 91 2022-07-29 09:00:32,219 INFO spawned: 'queuecleanupworker' with pid 93 2022-07-29 09:00:32,305 INFO spawned: 'repositoryactioncounter' with pid 94 2022-07-29 09:00:32,307 INFO spawned: 'repositorygcworker' with pid 96 2022-07-29 09:00:32,310 INFO spawned: 'securityscanningnotificationworker' with pid 97 2022-07-29 09:00:32,313 INFO spawned: 'securityworker' with pid 98 2022-07-29 09:00:32,405 INFO spawned: 'servicekey' with pid 105 2022-07-29 09:00:32,408 INFO spawned: 'storagereplication' with pid 117 2022-07-29 09:00:32,412 INFO spawned: 'teamsyncworker' with pid 118 2022-07-29 09:00:33,703 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: blobuploadcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: builder entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: buildlogsarchiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: chunkcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: expiredappspecifictokenworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: exportactionlogsworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: gcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: globalpromstats entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: gunicorn-registry entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: gunicorn-secscan entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: gunicorn-web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: jwtproxy entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: logrotateworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: manifestbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: memcache entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: namespacegcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: notificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: pushgateway entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: queuecleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: repositoryactioncounter entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: repositorygcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,704 INFO success: securityscanningnotificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,705 INFO success: securityworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,705 INFO success: servicekey entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,705 INFO success: storagereplication entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-07-29 09:00:33,705 INFO success: teamsyncworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) dnsmasq stderr | dnsmasq: started, version 2.79 cachesize 150 dnsmasq stderr | dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify dnsmasq stderr | dnsmasq: reading /etc/resolv.conf dnsmasq stderr | dnsmasq: using nameserver 12.0.0.10#53 dnsmasq stderr | dnsmasq: read /etc/hosts - 7 addresses jwtproxy stderr | time="2022-07-29T09:00:32Z" level=info msg="No claims verifiers specified, upstream should be configured to verify authorization" jwtproxy stderr | time="2022-07-29T09:00:32Z" level=info msg="Starting reverse proxy (Listening on 'unix:/tmp/jwtproxy_secscan.sock')" jwtproxy stderr | time="2022-07-29T09:00:32Z" level=info msg="Starting forward proxy (Listening on ':8081')" nginx stdout | 2022/07/29 09:00:32 [alert] 100#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 102#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 106#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 101#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 107#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 116#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 108#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 114#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 104#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 109#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 103#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 111#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 115#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 112#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 113#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/07/29 09:00:32 [alert] 110#0: setpriority(-10) failed (13: Permission denied) pushgateway stderr | level=info ts=2022-07-29T09:00:36.208Z caller=main.go:85 msg="starting pushgateway" version="(version=, branch=, revision=)" pushgateway stderr | level=info ts=2022-07-29T09:00:36.208Z caller=main.go:86 build_context="(go=go1.16.12, user=, date=)" pushgateway stderr | level=info ts=2022-07-29T09:00:36.305Z caller=main.go:139 listen_address=:9091 pushgateway stderr | level=info ts=2022-07-29T09:00:36.306Z caller=tls_config.go:191 msg="TLS is disabled." http2=false nginx stdout | 2022/07/29 09:00:42 [crit] 102#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 11.0.44.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "11.0.44.219:8080" nginx stdout | 2022/07/29 09:00:42 [crit] 102#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 11.0.44.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "11.0.44.219:8080" nginx stdout | 11.0.44.1 () - - [29/Jul/2022:09:00:42 +0000] "GET /health/instance HTTP/1.1" 502 173 "-" "kube-probe/1.21" (0.000 118 0.000 : 0.000) queuecleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) queuecleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) queuecleanupworker stderr | Failed to validate security scanner V2 configuration queuecleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) queuecleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() queuecleanupworker stdout | 2022-07-29 09:00:49,011 [93] [DEBUG] [workers.worker] Scheduling worker. queuecleanupworker stdout | 2022-07-29 09:00:49,012 [93] [INFO] [apscheduler.scheduler] Scheduler started queuecleanupworker stdout | 2022-07-29 09:00:49,012 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2022-07-29 09:00:49,012 [93] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added queuecleanupworker stdout | 2022-07-29 09:00:49,012 [93] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 11:43:03.013809 queuecleanupworker stdout | 2022-07-29 09:00:49,215 [93] [INFO] [apscheduler.scheduler] Added job "QueueCleanupWorker._cleanup_queue" to job store "default" queuecleanupworker stdout | 2022-07-29 09:00:49,303 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2022-07-29 09:00:49,303 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 11:43:03.013809+00:00 (in 9733.709989 seconds) namespacegcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) namespacegcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) namespacegcworker stderr | Failed to validate security scanner V2 configuration namespacegcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) namespacegcworker stderr | self._legacy_model = NoopV2SecurityScanner() namespacegcworker stdout | 2022-07-29 09:00:49,814 [83] [DEBUG] [__main__] Starting namespace GC worker namespacegcworker stdout | 2022-07-29 09:00:49,912 [83] [DEBUG] [workers.worker] Scheduling worker. namespacegcworker stdout | 2022-07-29 09:00:49,913 [83] [INFO] [apscheduler.scheduler] Scheduler started storagereplication stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) storagereplication stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) storagereplication stderr | Failed to validate security scanner V2 configuration storagereplication stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) storagereplication stderr | self._legacy_model = NoopV2SecurityScanner() namespacegcworker stdout | 2022-07-29 09:00:49,915 [83] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-07-29 09:00:49,915 [83] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added namespacegcworker stdout | 2022-07-29 09:00:49,915 [83] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:35.916918 storagereplication stdout | 2022-07-29 09:00:50,107 [117] [DEBUG] [__main__] Full storage replication disabled; skipping namespacegcworker stdout | 2022-07-29 09:00:50,208 [83] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" namespacegcworker stdout | 2022-07-29 09:00:50,208 [83] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:30.209906 namespacegcworker stdout | 2022-07-29 09:00:50,209 [83] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" namespacegcworker stdout | 2022-07-29 09:00:50,209 [83] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:14.210208 namespacegcworker stdout | 2022-07-29 09:00:50,209 [83] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" namespacegcworker stdout | 2022-07-29 09:00:50,209 [83] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-07-29 09:00:50,210 [83] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:14.210208+00:00 (in 24.000240 seconds) globalpromstats stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) globalpromstats stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) globalpromstats stderr | Failed to validate security scanner V2 configuration globalpromstats stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) globalpromstats stderr | self._legacy_model = NoopV2SecurityScanner() builder stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) builder stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) builder stderr | Failed to validate security scanner V2 configuration builder stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) builder stderr | self._legacy_model = NoopV2SecurityScanner() globalpromstats stdout | 2022-07-29 09:00:50,514 [71] [DEBUG] [workers.worker] Scheduling worker. globalpromstats stdout | 2022-07-29 09:00:50,515 [71] [INFO] [apscheduler.scheduler] Scheduler started globalpromstats stdout | 2022-07-29 09:00:50,804 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2022-07-29 09:00:50,804 [71] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:21:03.805764 globalpromstats stdout | 2022-07-29 09:00:50,805 [71] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added builder stdout | 2022-07-29 09:00:51,004 [64] [DEBUG] [__main__] Building is disabled. Please enable the feature flag globalpromstats stdout | 2022-07-29 09:00:51,010 [71] [INFO] [apscheduler.scheduler] Added job "GlobalPrometheusStatsWorker._try_report_stats" to job store "default" globalpromstats stdout | 2022-07-29 09:00:51,011 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2022-07-29 09:00:51,011 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:21:03.805764+00:00 (in 1212.794276 seconds) chunkcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) chunkcleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) chunkcleanupworker stderr | Failed to validate security scanner V2 configuration chunkcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) chunkcleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() chunkcleanupworker stdout | 2022-07-29 09:00:52,115 [66] [DEBUG] [__main__] Swift storage not detected; sleeping buildlogsarchiver stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) buildlogsarchiver stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) buildlogsarchiver stderr | Failed to validate security scanner V2 configuration buildlogsarchiver stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) buildlogsarchiver stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-web stdout | 2022-07-29 09:00:53,306 [74] [INFO] [data.registry_model] =============================== gunicorn-web stdout | 2022-07-29 09:00:53,306 [74] [INFO] [data.registry_model] Using registry model `` gunicorn-web stdout | 2022-07-29 09:00:53,306 [74] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2022-07-29 09:00:53,313 [73] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2022-07-29 09:00:53,313 [73] [INFO] [data.registry_model] Using registry model `` gunicorn-secscan stdout | 2022-07-29 09:00:53,314 [73] [INFO] [data.registry_model] =============================== exportactionlogsworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) exportactionlogsworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) exportactionlogsworker stderr | Failed to validate security scanner V2 configuration exportactionlogsworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) exportactionlogsworker stderr | self._legacy_model = NoopV2SecurityScanner() buildlogsarchiver stdout | 2022-07-29 09:00:53,904 [65] [DEBUG] [workers.worker] Scheduling worker. buildlogsarchiver stdout | 2022-07-29 09:00:53,905 [65] [INFO] [apscheduler.scheduler] Scheduler started buildlogsarchiver stdout | 2022-07-29 09:00:53,909 [65] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-07-29 09:00:53,910 [65] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added buildlogsarchiver stdout | 2022-07-29 09:00:53,910 [65] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:09.911795 buildlogsarchiver stdout | 2022-07-29 09:00:54,209 [65] [INFO] [apscheduler.scheduler] Added job "ArchiveBuildLogsWorker._archive_redis_buildlogs" to job store "default" buildlogsarchiver stdout | 2022-07-29 09:00:54,210 [65] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-07-29 09:00:54,210 [65] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:09.911795+00:00 (in 15.701336 seconds) gcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gcworker stderr | Failed to validate security scanner V2 configuration gcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gcworker stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-secscan stdout | 2022-07-29 09:00:55,012 [73] [DEBUG] [app] Loading default config. gunicorn-secscan stdout | 2022-07-29 09:00:55,012 [73] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml securityscanningnotificationworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityscanningnotificationworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) securityscanningnotificationworker stderr | Failed to validate security scanner V2 configuration securityscanningnotificationworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityscanningnotificationworker stderr | self._legacy_model = NoopV2SecurityScanner() gcworker stdout | 2022-07-29 09:00:55,106 [70] [DEBUG] [workers.worker] Scheduling worker. gcworker stdout | 2022-07-29 09:00:55,107 [70] [INFO] [apscheduler.scheduler] Scheduler started gunicorn-secscan stdout | 2022-07-29 09:00:55,108 [73] [DEBUG] [app] Loaded config gunicorn-secscan stdout | 2022-07-29 09:00:55,109 [73] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-web stdout | 2022-07-29 09:00:55,106 [74] [DEBUG] [app] Loading default config. gunicorn-web stdout | 2022-07-29 09:00:55,106 [74] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gcworker stdout | 2022-07-29 09:00:55,112 [70] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-07-29 09:00:55,112 [70] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added gcworker stdout | 2022-07-29 09:00:55,112 [70] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:18.113307 gunicorn-web stdout | 2022-07-29 09:00:55,114 [74] [DEBUG] [app] Loaded config gunicorn-web stdout | 2022-07-29 09:00:55,115 [74] [INFO] [util.ipresolver] Loading AWS IP ranges from disk securityscanningnotificationworker stdout | 2022-07-29 09:00:55,313 [97] [DEBUG] [__main__] Starting security scanning notification worker gunicorn-secscan stdout | 2022-07-29 09:00:55,315 [73] [DEBUG] [util.ipresolver] Building AWS IP ranges gunicorn-web stdout | 2022-07-29 09:00:55,405 [74] [DEBUG] [util.ipresolver] Building AWS IP ranges securityscanningnotificationworker stdout | 2022-07-29 09:00:55,506 [97] [DEBUG] [workers.worker] Scheduling worker. securityscanningnotificationworker stdout | 2022-07-29 09:00:55,507 [97] [INFO] [apscheduler.scheduler] Scheduler started securityscanningnotificationworker stdout | 2022-07-29 09:00:55,507 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-07-29 09:00:55,507 [97] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityscanningnotificationworker stdout | 2022-07-29 09:00:55,507 [97] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:25.508914 teamsyncworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) teamsyncworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) teamsyncworker stderr | Failed to validate security scanner V2 configuration teamsyncworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) teamsyncworker stderr | self._legacy_model = NoopV2SecurityScanner() gcworker stdout | 2022-07-29 09:00:55,604 [70] [INFO] [apscheduler.scheduler] Added job "GarbageCollectionWorker._garbage_collection_repos" to job store "default" gcworker stdout | 2022-07-29 09:00:55,613 [70] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-07-29 09:00:55,614 [70] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:18.113307+00:00 (in 22.499206 seconds) securityscanningnotificationworker stdout | 2022-07-29 09:00:55,715 [97] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" securityscanningnotificationworker stdout | 2022-07-29 09:00:55,715 [97] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:03:02.716749 securityscanningnotificationworker stdout | 2022-07-29 09:00:55,716 [97] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" securityscanningnotificationworker stdout | 2022-07-29 09:00:55,716 [97] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:45.717043 securityscanningnotificationworker stdout | 2022-07-29 09:00:55,716 [97] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" securityscanningnotificationworker stdout | 2022-07-29 09:00:55,716 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-07-29 09:00:55,716 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:25.508914+00:00 (in 29.792557 seconds) teamsyncworker stdout | 2022-07-29 09:00:55,807 [118] [DEBUG] [__main__] Team syncing is disabled; sleeping servicekey stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) servicekey stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) servicekey stderr | Failed to validate security scanner V2 configuration servicekey stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) servicekey stderr | self._legacy_model = NoopV2SecurityScanner() repositoryactioncounter stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositoryactioncounter stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) repositoryactioncounter stderr | Failed to validate security scanner V2 configuration repositoryactioncounter stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositoryactioncounter stderr | self._legacy_model = NoopV2SecurityScanner() servicekey stdout | 2022-07-29 09:00:56,512 [105] [DEBUG] [workers.worker] Scheduling worker. servicekey stdout | 2022-07-29 09:00:56,513 [105] [INFO] [apscheduler.scheduler] Scheduler started repositoryactioncounter stdout | 2022-07-29 09:00:56,513 [94] [DEBUG] [workers.worker] Scheduling worker. servicekey stdout | 2022-07-29 09:00:56,513 [105] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2022-07-29 09:00:56,513 [105] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositoryactioncounter stdout | 2022-07-29 09:00:56,514 [94] [INFO] [apscheduler.scheduler] Scheduler started servicekey stdout | 2022-07-29 09:00:56,513 [105] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:15:59.514942 repositoryactioncounter stdout | 2022-07-29 09:00:56,514 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2022-07-29 09:00:56,514 [94] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositoryactioncounter stdout | 2022-07-29 09:00:56,516 [94] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 10:30:47.517391 repositoryactioncounter stdout | 2022-07-29 09:00:56,719 [94] [INFO] [apscheduler.scheduler] Added job "RepositoryActionCountWorker._run_counting" to job store "default" repositoryactioncounter stdout | 2022-07-29 09:00:56,719 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2022-07-29 09:00:56,719 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 10:30:47.517391+00:00 (in 5390.797802 seconds) servicekey stdout | 2022-07-29 09:00:56,804 [105] [INFO] [apscheduler.scheduler] Added job "ServiceKeyWorker._refresh_service_key" to job store "default" servicekey stdout | 2022-07-29 09:00:56,804 [105] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2022-07-29 09:00:56,805 [105] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:15:59.514942+00:00 (in 902.710018 seconds) securityworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) securityworker stderr | Failed to validate security scanner V2 configuration securityworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityworker stderr | self._legacy_model = NoopV2SecurityScanner() expiredappspecifictokenworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) expiredappspecifictokenworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) expiredappspecifictokenworker stderr | Failed to validate security scanner V2 configuration expiredappspecifictokenworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) expiredappspecifictokenworker stderr | self._legacy_model = NoopV2SecurityScanner() exportactionlogsworker stdout | 2022-07-29 09:00:57,113 [69] [DEBUG] [__main__] Starting export action logs worker expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,205 [68] [DEBUG] [__main__] Starting expired app specific token GC worker exportactionlogsworker stdout | 2022-07-29 09:00:57,211 [69] [DEBUG] [workers.worker] Scheduling worker. exportactionlogsworker stdout | 2022-07-29 09:00:57,212 [69] [INFO] [apscheduler.scheduler] Scheduler started exportactionlogsworker stdout | 2022-07-29 09:00:57,212 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-07-29 09:00:57,213 [69] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added exportactionlogsworker stdout | 2022-07-29 09:00:57,213 [69] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:11.214346 securityworker stdout | 2022-07-29 09:00:57,310 [98] [DEBUG] [workers.worker] Scheduling worker. securityworker stdout | 2022-07-29 09:00:57,311 [98] [INFO] [apscheduler.scheduler] Scheduler started securityworker stdout | 2022-07-29 09:00:57,311 [98] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-07-29 09:00:57,312 [98] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityworker stdout | 2022-07-29 09:00:57,312 [98] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:18.313117 expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,312 [68] [DEBUG] [__main__] Found expiration window: 1d expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,315 [68] [DEBUG] [workers.worker] Scheduling worker. expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,316 [68] [INFO] [apscheduler.scheduler] Scheduler started expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,317 [68] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,317 [68] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,317 [68] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:04:47.318279 exportactionlogsworker stdout | 2022-07-29 09:00:57,413 [69] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" exportactionlogsworker stdout | 2022-07-29 09:00:57,413 [69] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:12.414352 exportactionlogsworker stdout | 2022-07-29 09:00:57,413 [69] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" exportactionlogsworker stdout | 2022-07-29 09:00:57,413 [69] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:12.414631 exportactionlogsworker stdout | 2022-07-29 09:00:57,413 [69] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" exportactionlogsworker stdout | 2022-07-29 09:00:57,414 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-07-29 09:00:57,414 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:11.214346+00:00 (in 13.799882 seconds) repositorygcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositorygcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) repositorygcworker stderr | Failed to validate security scanner V2 configuration repositorygcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositorygcworker stderr | self._legacy_model = NoopV2SecurityScanner() securityworker stdout | 2022-07-29 09:00:57,510 [98] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_in_scanner" to job store "default" securityworker stdout | 2022-07-29 09:00:57,510 [98] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:20.511785 securityworker stdout | 2022-07-29 09:00:57,511 [98] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_recent_manifests_in_scanner" to job store "default" securityworker stdout | 2022-07-29 09:00:57,511 [98] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-07-29 09:00:57,511 [98] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:18.313117+00:00 (in 20.801763 seconds) repositorygcworker stdout | 2022-07-29 09:00:57,523 [96] [DEBUG] [__main__] Starting repository GC worker blobuploadcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) blobuploadcleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) blobuploadcleanupworker stderr | Failed to validate security scanner V2 configuration blobuploadcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) blobuploadcleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() repositorygcworker stdout | 2022-07-29 09:00:57,618 [96] [DEBUG] [workers.worker] Scheduling worker. repositorygcworker stdout | 2022-07-29 09:00:57,619 [96] [INFO] [apscheduler.scheduler] Scheduler started repositorygcworker stdout | 2022-07-29 09:00:57,619 [96] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,620 [68] [INFO] [apscheduler.scheduler] Added job "ExpiredAppSpecificTokenWorker._gc_expired_tokens" to job store "default" repositorygcworker stdout | 2022-07-29 09:00:57,620 [96] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositorygcworker stdout | 2022-07-29 09:00:57,620 [96] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:45.620931 expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,621 [68] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2022-07-29 09:00:57,621 [68] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:04:47.318279+00:00 (in 229.697040 seconds) nginx stdout | 2022/07/29 09:00:57 [crit] 102#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 11.0.44.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "11.0.44.219:8080" nginx stdout | 2022/07/29 09:00:57 [crit] 102#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 11.0.44.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "11.0.44.219:8080" nginx stdout | 11.0.44.1 () - - [29/Jul/2022:09:00:57 +0000] "GET /health/instance HTTP/1.1" 502 173 "-" "kube-probe/1.21" (0.000 118 0.000 : 0.000) manifestbackfillworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) manifestbackfillworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) manifestbackfillworker stderr | Failed to validate security scanner V2 configuration manifestbackfillworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) manifestbackfillworker stderr | self._legacy_model = NoopV2SecurityScanner() blobuploadcleanupworker stdout | 2022-07-29 09:00:57,807 [63] [DEBUG] [workers.worker] Scheduling worker. repositorygcworker stdout | 2022-07-29 09:00:57,807 [96] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" repositorygcworker stdout | 2022-07-29 09:00:57,807 [96] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:02:44.808222 repositorygcworker stdout | 2022-07-29 09:00:57,807 [96] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" repositorygcworker stdout | 2022-07-29 09:00:57,807 [96] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:00:58.808544 repositorygcworker stdout | 2022-07-29 09:00:57,807 [96] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" blobuploadcleanupworker stdout | 2022-07-29 09:00:57,807 [63] [INFO] [apscheduler.scheduler] Scheduler started repositorygcworker stdout | 2022-07-29 09:00:57,807 [96] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-07-29 09:00:57,808 [96] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:00:58.808544+00:00 (in 1.000517 seconds) blobuploadcleanupworker stdout | 2022-07-29 09:00:57,808 [63] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2022-07-29 09:00:57,808 [63] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added blobuploadcleanupworker stdout | 2022-07-29 09:00:57,808 [63] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:14:42.809886 manifestbackfillworker stdout | 2022-07-29 09:00:57,906 [77] [DEBUG] [workers.worker] Scheduling worker. manifestbackfillworker stdout | 2022-07-29 09:00:57,907 [77] [INFO] [apscheduler.scheduler] Scheduler started manifestbackfillworker stdout | 2022-07-29 09:00:57,908 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2022-07-29 09:00:57,908 [77] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added manifestbackfillworker stdout | 2022-07-29 09:00:57,908 [77] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:33:35.909358 blobuploadcleanupworker stdout | 2022-07-29 09:00:57,912 [63] [INFO] [apscheduler.scheduler] Added job "BlobUploadCleanupWorker._try_cleanup_uploads" to job store "default" blobuploadcleanupworker stdout | 2022-07-29 09:00:57,912 [63] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2022-07-29 09:00:57,912 [63] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:14:42.809886+00:00 (in 824.897370 seconds) manifestbackfillworker stdout | 2022-07-29 09:00:57,938 [77] [INFO] [apscheduler.scheduler] Added job "ManifestBackfillWorker._backfill_manifests" to job store "default" manifestbackfillworker stdout | 2022-07-29 09:00:57,940 [77] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2022-07-29 09:00:57,940 [77] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:33:35.909358+00:00 (in 1957.968644 seconds) gunicorn-web stdout | 2022-07-29 09:00:58,326 [74] [DEBUG] [util.ipresolver] Finished building AWS IP ranges gunicorn-web stdout | 2022-07-29 09:00:58,329 [74] [DEBUG] [botocore.hooks] Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane gunicorn-web stdout | 2022-07-29 09:00:58,331 [74] [DEBUG] [botocore.hooks] Changing event name from before-call.apigateway to before-call.api-gateway gunicorn-web stdout | 2022-07-29 09:00:58,331 [74] [DEBUG] [botocore.hooks] Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict gunicorn-web stdout | 2022-07-29 09:00:58,332 [74] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration gunicorn-web stdout | 2022-07-29 09:00:58,332 [74] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.route53 to before-parameter-build.route-53 gunicorn-web stdout | 2022-07-29 09:00:58,333 [74] [DEBUG] [botocore.hooks] Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search gunicorn-web stdout | 2022-07-29 09:00:58,333 [74] [DEBUG] [botocore.hooks] Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section gunicorn-web stdout | 2022-07-29 09:00:58,335 [74] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask gunicorn-web stdout | 2022-07-29 09:00:58,335 [74] [DEBUG] [botocore.hooks] Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section gunicorn-web stdout | 2022-07-29 09:00:58,336 [74] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search gunicorn-web stdout | 2022-07-29 09:00:58,336 [74] [DEBUG] [botocore.hooks] Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section gunicorn-secscan stdout | 2022-07-29 09:00:58,337 [73] [DEBUG] [util.ipresolver] Finished building AWS IP ranges gunicorn-secscan stdout | 2022-07-29 09:00:58,404 [73] [DEBUG] [botocore.hooks] Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane gunicorn-secscan stdout | 2022-07-29 09:00:58,406 [73] [DEBUG] [botocore.hooks] Changing event name from before-call.apigateway to before-call.api-gateway gunicorn-secscan stdout | 2022-07-29 09:00:58,406 [73] [DEBUG] [botocore.hooks] Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict gunicorn-secscan stdout | 2022-07-29 09:00:58,407 [73] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration gunicorn-secscan stdout | 2022-07-29 09:00:58,407 [73] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.route53 to before-parameter-build.route-53 gunicorn-secscan stdout | 2022-07-29 09:00:58,408 [73] [DEBUG] [botocore.hooks] Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search gunicorn-secscan stdout | 2022-07-29 09:00:58,408 [73] [DEBUG] [botocore.hooks] Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section gunicorn-secscan stdout | 2022-07-29 09:00:58,410 [73] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask gunicorn-secscan stdout | 2022-07-29 09:00:58,410 [73] [DEBUG] [botocore.hooks] Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section gunicorn-secscan stdout | 2022-07-29 09:00:58,410 [73] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search gunicorn-secscan stdout | 2022-07-29 09:00:58,410 [73] [DEBUG] [botocore.hooks] Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section gunicorn-web stdout | 2022-07-29 09:00:58,512 [74] [DEBUG] [data.database] Configuring database gunicorn-web stdout | 2022-07-29 09:00:58,513 [74] [INFO] [data.database] Connection pooling disabled for mysql+pymysql gunicorn-web stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-web stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [INFO] [data.secscan_model] =============================== gunicorn-web stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-web stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [INFO] [data.secscan_model] =============================== gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [INFO] [data.logs_model] =============================== gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [INFO] [data.logs_model] Using logs model `` gunicorn-web stdout | 2022-07-29 09:00:58,514 [74] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2022-07-29 09:00:58,529 [72] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2022-07-29 09:00:58,530 [72] [INFO] [data.registry_model] Using registry model `` gunicorn-registry stdout | 2022-07-29 09:00:58,530 [72] [INFO] [data.registry_model] =============================== notificationworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) notificationworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) notificationworker stderr | Failed to validate security scanner V2 configuration notificationworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) notificationworker stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-secscan stdout | 2022-07-29 09:00:58,622 [73] [DEBUG] [data.database] Configuring database gunicorn-secscan stdout | 2022-07-29 09:00:58,623 [73] [INFO] [data.database] Connection pooling disabled for mysql+pymysql gunicorn-secscan stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-secscan stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-secscan stdout | 2022-07-29 09:00:58,624 [73] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-secscan stdout | 2022-07-29 09:00:58,624 [73] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-secscan stdout | 2022-07-29 09:00:58,625 [73] [INFO] [data.secscan_model] =============================== gunicorn-secscan stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-secscan stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-secscan stdout | 2022-07-29 09:00:58,625 [73] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-secscan stdout | 2022-07-29 09:00:58,625 [73] [INFO] [data.secscan_model] =============================== gunicorn-secscan stdout | 2022-07-29 09:00:58,625 [73] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-secscan stdout | 2022-07-29 09:00:58,625 [73] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2022-07-29 09:00:58,625 [73] [INFO] [data.logs_model] Using logs model `` gunicorn-secscan stdout | 2022-07-29 09:00:58,625 [73] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2022-07-29 09:00:58,635 [73] [DEBUG] [__config__] Starting secscan gunicorn with 2 workers and gevent worker class notificationworker stdout | 2022-07-29 09:00:58,729 [90] [DEBUG] [workers.worker] Scheduling worker. notificationworker stdout | 2022-07-29 09:00:58,729 [90] [INFO] [apscheduler.scheduler] Scheduler started notificationworker stdout | 2022-07-29 09:00:58,730 [90] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-07-29 09:00:58,730 [90] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:08.731356 notificationworker stdout | 2022-07-29 09:00:58,730 [90] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositorygcworker stdout | 2022-07-29 09:00:58,808 [96] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-07-29 09:00:58,809 [96] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-07-29 09:00:58 UTC)" (scheduled at 2022-07-29 09:00:58.808544+00:00) repositorygcworker stdout | 2022-07-29 09:00:58,809 [96] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-07-29 09:00:58,809 [96] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-07-29 09:00:58 UTC)" executed successfully repositorygcworker stdout | 2022-07-29 09:00:58,810 [96] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:45.620931+00:00 (in 46.811909 seconds) notificationworker stdout | 2022-07-29 09:00:58,821 [90] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" notificationworker stdout | 2022-07-29 09:00:58,821 [90] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:05:19.822509 notificationworker stdout | 2022-07-29 09:00:58,821 [90] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" notificationworker stdout | 2022-07-29 09:00:58,821 [90] [DEBUG] [workers.worker] First run scheduled for 2022-07-29 09:01:11.822873 notificationworker stdout | 2022-07-29 09:00:58,822 [90] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" notificationworker stdout | 2022-07-29 09:00:58,822 [90] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-07-29 09:00:58,822 [90] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:08.731356+00:00 (in 9.908757 seconds) gunicorn-registry stdout | 2022-07-29 09:00:58,956 [72] [DEBUG] [app] Loading default config. gunicorn-registry stdout | 2022-07-29 09:00:58,956 [72] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml logrotateworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) logrotateworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-registry stdout | 2022-07-29 09:00:58,962 [72] [DEBUG] [app] Loaded config logrotateworker stderr | Failed to validate security scanner V2 configuration logrotateworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) logrotateworker stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-registry stdout | 2022-07-29 09:00:58,963 [72] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-registry stdout | 2022-07-29 09:00:59,013 [72] [DEBUG] [util.ipresolver] Building AWS IP ranges logrotateworker stdout | 2022-07-29 09:00:59,112 [76] [DEBUG] [__main__] Action log rotation worker not enabled; skipping gunicorn-web stdout | 2022-07-29 09:00:59,326 [74] [DEBUG] [__config__] Starting web gunicorn with 4 workers and gevent worker class gunicorn-registry stdout | 2022-07-29 09:00:59,650 [72] [DEBUG] [util.ipresolver] Finished building AWS IP ranges gunicorn-registry stdout | 2022-07-29 09:00:59,652 [72] [DEBUG] [botocore.hooks] Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane gunicorn-registry stdout | 2022-07-29 09:00:59,653 [72] [DEBUG] [botocore.hooks] Changing event name from before-call.apigateway to before-call.api-gateway gunicorn-registry stdout | 2022-07-29 09:00:59,654 [72] [DEBUG] [botocore.hooks] Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict gunicorn-registry stdout | 2022-07-29 09:00:59,655 [72] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration gunicorn-registry stdout | 2022-07-29 09:00:59,655 [72] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.route53 to before-parameter-build.route-53 gunicorn-registry stdout | 2022-07-29 09:00:59,655 [72] [DEBUG] [botocore.hooks] Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search gunicorn-registry stdout | 2022-07-29 09:00:59,656 [72] [DEBUG] [botocore.hooks] Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section gunicorn-registry stdout | 2022-07-29 09:00:59,657 [72] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask gunicorn-registry stdout | 2022-07-29 09:00:59,658 [72] [DEBUG] [botocore.hooks] Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section gunicorn-registry stdout | 2022-07-29 09:00:59,658 [72] [DEBUG] [botocore.hooks] Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search gunicorn-registry stdout | 2022-07-29 09:00:59,658 [72] [DEBUG] [botocore.hooks] Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section gunicorn-registry stdout | 2022-07-29 09:00:59,705 [72] [DEBUG] [data.database] Configuring database gunicorn-registry stdout | 2022-07-29 09:00:59,706 [72] [INFO] [data.database] Connection pooling enabled for mysql+pymysql; stale timeout: None; max connection count: None gunicorn-registry stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-registry stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-registry stdout | 2022-07-29 09:00:59,706 [72] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-registry stdout | 2022-07-29 09:00:59,706 [72] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-registry stdout | 2022-07-29 09:00:59,707 [72] [INFO] [data.secscan_model] =============================== gunicorn-registry stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-registry stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-registry stdout | 2022-07-29 09:00:59,707 [72] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-registry stdout | 2022-07-29 09:00:59,707 [72] [INFO] [data.secscan_model] =============================== gunicorn-registry stdout | 2022-07-29 09:00:59,707 [72] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-registry stdout | 2022-07-29 09:00:59,707 [72] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2022-07-29 09:00:59,707 [72] [INFO] [data.logs_model] Using logs model `` gunicorn-registry stdout | 2022-07-29 09:00:59,707 [72] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2022-07-29 09:00:59,823 [72] [DEBUG] [__config__] Starting registry gunicorn with 8 workers and gevent worker class notificationworker stdout | 2022-07-29 09:01:08,731 [90] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-07-29 09:01:08,733 [90] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-07-29 09:01:08 UTC)" (scheduled at 2022-07-29 09:01:08.731356+00:00) notificationworker stdout | 2022-07-29 09:01:08,733 [90] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-07-29 09:01:08,734 [90] [DEBUG] [peewee] ('SELECT `t1`.`id`, `t1`.`queue_name`, `t1`.`body`, `t1`.`available_after`, `t1`.`available`, `t1`.`processing_expires`, `t1`.`retries_remaining`, `t1`.`state_id` FROM `queueitem` AS `t1` INNER JOIN (SELECT `t1`.`id` FROM `queueitem` AS `t1` WHERE ((((`t1`.`available_after` <= %s) AND ((`t1`.`available` = %s) OR (`t1`.`processing_expires` <= %s))) AND (`t1`.`retries_remaining` > %s)) AND (`t1`.`queue_name` LIKE %s)) LIMIT %s) AS `j1` ON (`t1`.`id` = `j1`.`id`) ORDER BY Rand() LIMIT %s OFFSET %s', [datetime.datetime(2022, 7, 29, 9, 1, 8, 733416), True, datetime.datetime(2022, 7, 29, 9, 1, 8, 733416), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-07-29 09:01:08,734 [90] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:11.822873+00:00 (in 3.091017 seconds) notificationworker stdout | 2022-07-29 09:01:08,765 [90] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-07-29 09:01:08,765 [90] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-07-29 09:01:08,766 [90] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-07-29 09:01:18 UTC)" executed successfully buildlogsarchiver stdout | 2022-07-29 09:01:09,912 [65] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-07-29 09:01:09,912 [65] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-07-29 09:01:09 UTC)" (scheduled at 2022-07-29 09:01:09.911795+00:00) buildlogsarchiver stdout | 2022-07-29 09:01:09,914 [65] [DEBUG] [peewee] ('SELECT `candidates`.`id` FROM (SELECT `t1`.`id` FROM `repositorybuild` AS `t1` WHERE (((`t1`.`phase` IN (%s, %s, %s)) OR (`t1`.`started` < %s)) AND (`t1`.`logs_archived` = %s)) LIMIT %s) AS `candidates` ORDER BY Rand() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 7, 14, 9, 1, 9, 913206), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-07-29 09:01:09,914 [65] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:39.911795+00:00 (in 29.999544 seconds) buildlogsarchiver stdout | 2022-07-29 09:01:09,951 [65] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-07-29 09:01:09,951 [65] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-07-29 09:01:09,951 [65] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-07-29 09:01:39 UTC)" executed successfully exportactionlogsworker stdout | 2022-07-29 09:01:11,214 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-07-29 09:01:11,215 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-07-29 09:01:11 UTC)" (scheduled at 2022-07-29 09:01:11.214346+00:00) exportactionlogsworker stdout | 2022-07-29 09:01:11,215 [69] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-07-29 09:01:11,216 [69] [DEBUG] [peewee] ('SELECT `t1`.`id`, `t1`.`queue_name`, `t1`.`body`, `t1`.`available_after`, `t1`.`available`, `t1`.`processing_expires`, `t1`.`retries_remaining`, `t1`.`state_id` FROM `queueitem` AS `t1` INNER JOIN (SELECT `t1`.`id` FROM `queueitem` AS `t1` WHERE ((((`t1`.`available_after` <= %s) AND ((`t1`.`available` = %s) OR (`t1`.`processing_expires` <= %s))) AND (`t1`.`retries_remaining` > %s)) AND (`t1`.`queue_name` LIKE %s)) LIMIT %s) AS `j1` ON (`t1`.`id` = `j1`.`id`) ORDER BY Rand() LIMIT %s OFFSET %s', [datetime.datetime(2022, 7, 29, 9, 1, 11, 215674), True, datetime.datetime(2022, 7, 29, 9, 1, 11, 215674), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-07-29 09:01:11,217 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:12.414352+00:00 (in 1.199494 seconds) exportactionlogsworker stdout | 2022-07-29 09:01:11,242 [69] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-07-29 09:01:11,242 [69] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-07-29 09:01:11,242 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-07-29 09:02:11 UTC)" executed successfully notificationworker stdout | 2022-07-29 09:01:11,826 [90] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-07-29 09:01:11,826 [90] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:18.731356+00:00 (in 6.904981 seconds) notificationworker stdout | 2022-07-29 09:01:11,826 [90] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-07-29 09:02:11 UTC)" (scheduled at 2022-07-29 09:01:11.822873+00:00) notificationworker stdout | 2022-07-29 09:01:11,827 [90] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-07-29 09:01:11,827 [90] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-07-29 09:02:11 UTC)" executed successfully namespacegcworker stdout | 2022-07-29 09:01:12,350 [83] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'namespacegcworker.py', 'pid': '83'} exportactionlogsworker stdout | 2022-07-29 09:01:12,416 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-07-29 09:01:12,417 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-07-29 09:06:12 UTC)" (scheduled at 2022-07-29 09:01:12.414352+00:00) exportactionlogsworker stdout | 2022-07-29 09:01:12,418 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT `t1`.`queue_name` FROM `queueitem` AS `t1` WHERE (((`t1`.`available` = %s) AND (`t1`.`processing_expires` > %s)) AND (`t1`.`queue_name` LIKE %s))) AS `_wrapped`', [False, datetime.datetime(2022, 7, 29, 9, 1, 12, 417966), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-07-29 09:01:12,419 [69] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-07-29 09:01:12 UTC)" (scheduled at 2022-07-29 09:01:12.414631+00:00) exportactionlogsworker stdout | 2022-07-29 09:01:12,419 [69] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-07-29 09:01:12,419 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:02:11.214346+00:00 (in 58.797102 seconds) exportactionlogsworker stdout | 2022-07-29 09:01:12,419 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-07-29 09:02:12 UTC)" executed successfully exportactionlogsworker stdout | 2022-07-29 09:01:12,444 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT `t1`.`queue_name` FROM `queueitem` AS `t1` WHERE ((((`t1`.`available_after` <= %s) AND ((`t1`.`available` = %s) OR (`t1`.`processing_expires` <= %s))) AND (`t1`.`retries_remaining` > %s)) AND (`t1`.`queue_name` LIKE %s))) AS `_wrapped`', [datetime.datetime(2022, 7, 29, 9, 1, 12, 417966), True, datetime.datetime(2022, 7, 29, 9, 1, 12, 417966), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-07-29 09:01:12,448 [69] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT `t1`.`queue_name` FROM `queueitem` AS `t1` WHERE (((((`t1`.`available_after` <= %s) AND ((`t1`.`available` = %s) OR (`t1`.`processing_expires` <= %s))) AND (`t1`.`retries_remaining` > %s)) AND (`t1`.`queue_name` LIKE %s)) AND NOT (`t1`.`queue_name` IN (SELECT `t1`.`queue_name` FROM `queueitem` AS `t1` WHERE (((`t1`.`available` = %s) AND (`t1`.`processing_expires` > %s)) AND (`t1`.`queue_name` LIKE %s)))))) AS `_wrapped`', [datetime.datetime(2022, 7, 29, 9, 1, 12, 417966), True, datetime.datetime(2022, 7, 29, 9, 1, 12, 417966), 0, 'exportactionlogs/%', False, datetime.datetime(2022, 7, 29, 9, 1, 12, 417966), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-07-29 09:01:12,452 [69] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-07-29 09:01:12,452 [69] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-07-29 09:06:12 UTC)" executed successfully globalpromstats stdout | 2022-07-29 09:01:12,612 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'globalpromstats.py', 'pid': '71'} gunicorn-web stdout | 2022-07-29 09:01:12,625 [239] [DEBUG] [app] Starting request: urn:request:94eddb1e-588a-48d1-ba09-3342e3c2ce9a (/health/instance) {'X-Forwarded-For': '11.0.44.1'} gunicorn-web stdout | 2022-07-29 09:01:12,633 [239] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 gunicorn-registry stdout | 2022-07-29 09:01:12,647 [245] [DEBUG] [app] Starting request: urn:request:02d8bef4-9527-40db-9b0a-dcbdd55baa45 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-registry stdout | 2022-07-29 09:01:12,650 [245] [DEBUG] [app] Ending request: urn:request:02d8bef4-9527-40db-9b0a-dcbdd55baa45 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:02d8bef4-9527-40db-9b0a-dcbdd55baa45', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'dca422c7', 'user-agent': 'python-requests/2.27.1'} gunicorn-registry stdout | 2022-07-29 09:01:12,651 [245] [INFO] [gunicorn.access] 127.0.0.1 - - [29/Jul/2022:09:01:12 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" nginx stdout | 127.0.0.1 () - - [29/Jul/2022:09:01:12 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.010 162 0.010) queuecleanupworker stdout | 2022-07-29 09:01:12,652 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'queuecleanupworker.py', 'pid': '93'} gunicorn-web stdout | 2022-07-29 09:01:12,652 [239] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-07-29 09:01:12,654 [239] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-07-29 09:01:12,655 [239] [DEBUG] [app] Starting request: urn:request:66bbc608-799c-40d0-afca-76f1874b6bd9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'} gunicorn-web stdout | 2022-07-29 09:01:12,657 [239] [DEBUG] [app] Ending request: urn:request:66bbc608-799c-40d0-afca-76f1874b6bd9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:66bbc608-799c-40d0-afca-76f1874b6bd9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '37a128c9', 'user-agent': 'python-requests/2.27.1'} nginx stdout | 127.0.0.1 () - - [29/Jul/2022:09:01:12 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.27.1" (0.003 159 0.003) gunicorn-web stdout | 2022-07-29 09:01:12,658 [239] [INFO] [gunicorn.access] 127.0.0.1 - - [29/Jul/2022:09:01:12 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.27.1" gunicorn-web stdout | 2022-07-29 09:01:12,658 [239] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-07-29 09:01:12,662 [239] [DEBUG] [peewee] ('SELECT `t1`.`id`, `t1`.`name`, `t1`.`kid`, `t1`.`service`, `t1`.`jwk`, `t1`.`metadata`, `t1`.`created_date`, `t1`.`expiration_date`, `t1`.`rotation_duration`, `t1`.`approval_id` FROM `servicekey` AS `t1` LEFT OUTER JOIN `servicekeyapproval` AS `t2` ON (`t1`.`approval_id` = `t2`.`id`) WHERE ((((NOT (`t1`.`approval_id` IS %s) AND ((`t1`.`expiration_date` > %s) OR (`t1`.`expiration_date` IS %s))) AND (`t1`.`service` = %s)) AND (NOT ((`t1`.`service` = %s) AND (`t1`.`expiration_date` <= %s)) OR NOT (((`t1`.`service` = %s) AND (`t1`.`approval_id` IS %s)) AND (`t1`.`created_date` <= %s)))) AND (NOT (`t1`.`expiration_date` <= %s) OR (`t1`.`expiration_date` IS %s)))', [None, datetime.datetime(2022, 7, 29, 9, 1, 12, 660151), None, 'quay', 'quay', datetime.datetime(2022, 7, 29, 9, 1, 12, 660299), 'quay', None, datetime.datetime(2022, 7, 28, 9, 1, 12, 660321), datetime.datetime(2022, 7, 22, 9, 1, 12, 660331), None]) gunicorn-web stdout | 2022-07-29 09:01:12,734 [239] [INFO] [data.database] Connection pooling disabled for mysql+pymysql gunicorn-web stdout | 2022-07-29 09:01:12,745 [239] [WARNING] [health.healthcheck] [FAILED HEALTH CHECK] {'services_expanded': {'registry_gunicorn': {'status': True}, 'web_gunicorn': {'status': True}, 'service_key': {'status': True}, 'disk_space': {'status': True}, 'database': {'status': False, 'failure': 'Could not connect to the database: (1045, "Access denied for user \'quayuser\'@\'mpcrdcocpqua03.pcrdc.prodpcr.rbi.org.in\' (using password: YES)")'}, 'auth': {'status': True}}, 'notes': [], 'is_testing': False, 'config_provider': 'k8s', 'local_service_key_id': '_qsAOUj_X55KJhZ4Cw--Hxq-be1q7_pCyvSl3ybw3po', 'hostname': 'quay-registry-quay-app-b9c495776-gxkjz'} gunicorn-web stdout | 2022-07-29 09:01:12,745 [239] [DEBUG] [app] Ending request: urn:request:94eddb1e-588a-48d1-ba09-3342e3c2ce9a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:94eddb1e-588a-48d1-ba09-3342e3c2ce9a', 'remote_addr': '11.0.44.1', 'http_method': 'GET', 'original_url': 'http://11.0.44.219/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '37a128c9', 'user-agent': 'kube-probe/1.21'} gunicorn-web stdout | 2022-07-29 09:01:12,745 [239] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 11.0.44.1 () - - [29/Jul/2022:09:01:12 +0000] "GET /health/instance HTTP/1.1" 503 153 "-" "kube-probe/1.21" (0.127 118 0.127) gunicorn-web stdout | 2022-07-29 09:01:12,746 [239] [INFO] [gunicorn.access] 11.0.44.1 - - [29/Jul/2022:09:01:12 +0000] "GET /health/instance HTTP/1.0" 503 153 "-" "kube-probe/1.21" chunkcleanupworker stdout | 2022-07-29 09:01:13,451 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'chunkcleanupworker.py', 'pid': '66'} builder stdout | 2022-07-29 09:01:13,543 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'builder.py', 'pid': '64'} repositoryactioncounter stdout | 2022-07-29 09:01:13,640 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'repositoryactioncounter.py', 'pid': '94'} storagereplication stdout | 2022-07-29 09:01:13,944 [117] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'storagereplication.py', 'pid': '117'} exportactionlogsworker stdout | 2022-07-29 09:01:14,061 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'exportactionlogsworker.py', 'pid': '69'} namespacegcworker stdout | 2022-07-29 09:01:14,210 [83] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-07-29 09:01:14,211 [83] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-07-29 09:01:14 UTC)" (scheduled at 2022-07-29 09:01:14.210208+00:00) namespacegcworker stdout | 2022-07-29 09:01:14,211 [83] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-07-29 09:01:14,211 [83] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-07-29 09:01:14 UTC)" executed successfully namespacegcworker stdout | 2022-07-29 09:01:14,213 [83] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:30.209906+00:00 (in 15.999147 seconds) buildlogsarchiver stdout | 2022-07-29 09:01:14,540 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'buildlogsarchiver.py', 'pid': '65'} securityscanningnotificationworker stdout | 2022-07-29 09:01:14,716 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'securityscanningnotificationworker.py', 'pid': '97'} servicekey stdout | 2022-07-29 09:01:16,222 [105] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'servicekeyworker.py', 'pid': '105'} gcworker stdout | 2022-07-29 09:01:17,731 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'gcworker.py', 'pid': '70'} teamsyncworker stdout | 2022-07-29 09:01:17,849 [118] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'teamsyncworker.py', 'pid': '118'} gcworker stdout | 2022-07-29 09:01:18,113 [70] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-07-29 09:01:18,114 [70] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-07-29 09:01:18 UTC)" (scheduled at 2022-07-29 09:01:18.113307+00:00) gcworker stdout | 2022-07-29 09:01:18,115 [70] [DEBUG] [peewee] ('SELECT DISTINCT `t1`.`removed_tag_expiration_s` FROM `user` AS `t1` LIMIT %s', [100]) gcworker stdout | 2022-07-29 09:01:18,115 [70] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:48.113307+00:00 (in 29.999485 seconds) gcworker stdout | 2022-07-29 09:01:18,141 [70] [DEBUG] [__main__] No GC policies found gcworker stdout | 2022-07-29 09:01:18,141 [70] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-07-29 09:01:18,141 [70] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-07-29 09:01:48 UTC)" executed successfully securityworker stdout | 2022-07-29 09:01:18,313 [98] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-07-29 09:01:18,314 [98] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-07-29 09:01:18 UTC)" (scheduled at 2022-07-29 09:01:18.313117+00:00) securityworker stdout | 2022-07-29 09:01:18,314 [98] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-07-29 09:01:18,314 [98] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-registry-clair-app.quay-registry.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-07-29 09:01:18,315 [98] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:20.511785+00:00 (in 2.198034 seconds) securityworker stdout | 2022-07-29 09:01:18,317 [98] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): quay-registry-clair-app.quay-registry.svc.cluster.local:80 securityworker stdout | 2022-07-29 09:01:18,323 [98] [DEBUG] [urllib3.connectionpool] http://quay-registry-clair-app.quay-registry.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-07-29 09:01:18,326 [98] [DEBUG] [peewee] ('SELECT Max(`t1`.`id`) FROM `manifest` AS `t1`', []) securityworker stdout | 2022-07-29 09:01:18,350 [98] [DEBUG] [peewee] ('SELECT Min(`t1`.`id`) FROM `manifest` AS `t1`', []) securityworker stdout | 2022-07-29 09:01:18,353 [98] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-07-29 09:01:18,353 [98] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-07-29 09:01:48 UTC)" executed successfully repositorygcworker stdout | 2022-07-29 09:01:18,356 [96] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'repositorygcworker.py', 'pid': '96'} notificationworker stdout | 2022-07-29 09:01:18,731 [90] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-07-29 09:01:18,732 [90] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:28.731356+00:00 (in 9.999273 seconds) notificationworker stdout | 2022-07-29 09:01:18,732 [90] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-07-29 09:01:28 UTC)" (scheduled at 2022-07-29 09:01:18.731356+00:00) notificationworker stdout | 2022-07-29 09:01:18,732 [90] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-07-29 09:01:18,734 [90] [DEBUG] [peewee] ('SELECT `t1`.`id`, `t1`.`queue_name`, `t1`.`body`, `t1`.`available_after`, `t1`.`available`, `t1`.`processing_expires`, `t1`.`retries_remaining`, `t1`.`state_id` FROM `queueitem` AS `t1` INNER JOIN (SELECT `t1`.`id` FROM `queueitem` AS `t1` WHERE ((((`t1`.`available_after` <= %s) AND ((`t1`.`available` = %s) OR (`t1`.`processing_expires` <= %s))) AND (`t1`.`retries_remaining` > %s)) AND (`t1`.`queue_name` LIKE %s)) LIMIT %s) AS `j1` ON (`t1`.`id` = `j1`.`id`) ORDER BY Rand() LIMIT %s OFFSET %s', [datetime.datetime(2022, 7, 29, 9, 1, 18, 732801), True, datetime.datetime(2022, 7, 29, 9, 1, 18, 732801), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-07-29 09:01:18,800 [90] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-07-29 09:01:18,800 [90] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-07-29 09:01:18,800 [90] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-07-29 09:01:28 UTC)" executed successfully expiredappspecifictokenworker stdout | 2022-07-29 09:01:19,642 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '68'} securityworker stdout | 2022-07-29 09:01:20,043 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-registry-quay-app-b9c495776-gxkjz', 'process_name': 'securityworker.py', 'pid': '98'} securityworker stdout | 2022-07-29 09:01:20,513 [98] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-07-29 09:01:20,514 [98] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-07-29 09:01:48.313117+00:00 (in 27.799219 seconds) securityworker stdout | 2022-07-29 09:01:20,514 [98] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-07-29 09:01:50 UTC)" (scheduled at 2022-07-29 09:01:20.511785+00:00) securityworker stdout | 2022-07-29 09:01:20,514 [98] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-07-29 09:01:20,514 [98] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ... securityworker stdout | 2022-07-29 09:01:20,516 [98] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-07-29 09:01:20,516 [98] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds securityworker stdout | 2022-07-29 09:01:20,516 [98] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-07-29 09:01:20,517 [98] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-07-29 09:01:20,517 [98] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-registry-clair-app.quay-registry.svc.cluster.local/indexer/api/v1/index_state securityworker stdout | 2022-07-29 09:01:20,518 [98] [DEBUG] [urllib3.connectionpool] http://quay-registry-clair-app.quay-registry.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-07-29 09:01:20,520 [98] [DEBUG] [peewee] ('SELECT Max(`t1`.`id`) FROM `manifest` AS `t1`', []) securityworker stdout | 2022-07-29 09:01:20,541 [98] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-07-29 09:01:20,541 [98] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop securityworker stdout | 2022-07-29 09:01:20,541 [98] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing securityworker stdout | 2022-07-29 09:01:20,541 [98] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped securityworker stdout | 2022-07-29 09:01:20,541 [98] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'. securityworker stdout | 2022-07-29 09:01:20,542 [98] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST securityworker stdout | 2022-07-29 09:01:20,542 [98] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-07-29 09:01:20,542 [98] [ERROR] [workers.worker] Operation raised exception securityworker stdout | Traceback (most recent call last): securityworker stdout | File "/quay-registry/workers/worker.py", line 87, in _operation_func securityworker stdout | return operation_func() securityworker stdout | File "/quay-registry/workers/securityworker/securityworker.py", line 45, in _index_recent_manifests_in_scanner securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/__init__.py", line 46, in perform_indexing_recent_manifests securityworker stdout | self._model.perform_indexing_recent_manifests(batch_size) securityworker stdout | File "/quay-registry/data/secscan_model/secscan_v4_model.py", line 251, in perform_indexing_recent_manifests securityworker stdout | start_index = max(end_index - batch_size, 1) securityworker stdout | TypeError: unsupported operand type(s) for -: 'NoneType' and 'int' securityworker stdout | 2022-07-29 09:01:20,543 [98] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2022-07-29 09:01:50 UTC)" executed successfully