__ __ / \ / \ ______ _ _ __ __ __ / /\ / /\ \ / __ \ | | | | / \ \ \ / / / / / / \ \ | | | | | | | | / /\ \ \ / \ \ \ \ / / | |__| | | |__| | / ____ \ | | \ \/ \ \/ / \_ ___/ \____/ /_/ \_\ |_| \__/ \__/ \ \__ \___\ by Red Hat Build, Store, and Distribute your Containers Running all default registry services without migration Running init script '/quay-registry/conf/init/certs_create.sh' Generating a RSA private key .............................................................................................++++ ..........................................................................++++ writing new private key to 'mitm-key.pem' ----- Running init script '/quay-registry/conf/init/certs_install.sh' Installing extra certificates found in /quay-registry/conf/stack/extra_ca_certs directory Running init script '/quay-registry/conf/init/copy_config_files.sh' Running init script '/quay-registry/conf/init/d_validate_config_bundle.sh' Validating Configuration time="2022-05-05T10:00:35Z" level=debug msg="Validating AccessSettings" time="2022-05-05T10:00:35Z" level=debug msg="Validating ActionLogArchiving" time="2022-05-05T10:00:35Z" level=debug msg="Validating AppTokenAuthentication" time="2022-05-05T10:00:35Z" level=debug msg="Validating BitbucketBuildTrigger" time="2022-05-05T10:00:35Z" level=debug msg="Validating BuildManager" time="2022-05-05T10:00:35Z" level=debug msg="Validating Database" time="2022-05-05T10:00:35Z" level=debug msg="Scheme: postgresql" time="2022-05-05T10:00:35Z" level=debug msg="Host: quay-quay-database:5432" time="2022-05-05T10:00:35Z" level=debug msg="Db: quay-quay-database" time="2022-05-05T10:00:35Z" level=debug msg="Params: " time="2022-05-05T10:00:35Z" level=debug msg="Including params " time="2022-05-05T10:00:35Z" level=debug msg="Pinging database at postgresql://quay-quay-database:30RGeUBC4ExbhV1WO362w7tbB9uRvKS2g1fD81hUxAIaqfdzlrxrE9yIoa7cDSDqbnlx5gnoMlnfUIkC@quay-quay-database:5432/quay-quay-database" plpgsql pg_trgm time="2022-05-05T10:00:35Z" level=debug msg="Validating DistributedStorage" time="2022-05-05T10:00:35Z" level=debug msg="Validating ElasticSearch" time="2022-05-05T10:00:35Z" level=debug msg="Validating Email" time="2022-05-05T10:00:35Z" level=debug msg="Validating GitHubBuildTrigger" time="2022-05-05T10:00:35Z" level=debug msg="Validating GitHubLogin" time="2022-05-05T10:00:35Z" level=debug msg="Validating GitLabBuildTrigger" time="2022-05-05T10:00:35Z" level=debug msg="Validating GoogleLogin" time="2022-05-05T10:00:35Z" level=debug msg="Validating HostSettings" time="2022-05-05T10:00:35Z" level=debug msg="Validating JWTAuthentication" time="2022-05-05T10:00:35Z" level=debug msg="Validating LDAP" time="2022-05-05T10:00:35Z" level=debug msg="Validating OIDC" time="2022-05-05T10:00:35Z" level=debug msg="Validating QuayDocumentation" time="2022-05-05T10:00:35Z" level=debug msg="Validating Redis" time="2022-05-05T10:00:35Z" level=debug msg="Address: quay-quay-redis:6379" time="2022-05-05T10:00:35Z" level=debug msg="Username: " time="2022-05-05T10:00:35Z" level=debug msg="Password Len: 0" time="2022-05-05T10:00:35Z" level=debug msg="Ssl: " time="2022-05-05T10:00:35Z" level=debug msg="Address: quay-quay-redis:6379" time="2022-05-05T10:00:35Z" level=debug msg="Username: " time="2022-05-05T10:00:35Z" level=debug msg="Password Len: 0" time="2022-05-05T10:00:35Z" level=debug msg="Ssl: " time="2022-05-05T10:00:35Z" level=debug msg="Validating RepoMirror" time="2022-05-05T10:00:35Z" level=debug msg="Validating SecurityScanner" time="2022-05-05T10:00:35Z" level=debug msg="Validating TeamSyncing" time="2022-05-05T10:00:35Z" level=debug msg="Validating TimeMachine" time="2022-05-05T10:00:35Z" level=debug msg="Validating UserVisibleSettings" +------------------------+-------+--------+ | Field Group | Error | Status | +------------------------+-------+--------+ | AccessSettings | - | 🟢 | +------------------------+-------+--------+ | ActionLogArchiving | - | 🟢 | +------------------------+-------+--------+ | AppTokenAuthentication | - | 🟢 | +------------------------+-------+--------+ | BitbucketBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | BuildManager | - | 🟢 | +------------------------+-------+--------+ | Database | - | 🟢 | +------------------------+-------+--------+ | DistributedStorage | - | 🟢 | +------------------------+-------+--------+ | ElasticSearch | - | 🟢 | +------------------------+-------+--------+ | Email | - | 🟢 | +------------------------+-------+--------+ | GitHubBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | GitHubLogin | - | 🟢 | +------------------------+-------+--------+ | GitLabBuildTrigger | - | 🟢 | +------------------------+-------+--------+ | GoogleLogin | - | 🟢 | +------------------------+-------+--------+ | HostSettings | - | 🟢 | +------------------------+-------+--------+ | JWTAuthentication | - | 🟢 | +------------------------+-------+--------+ | LDAP | - | 🟢 | +------------------------+-------+--------+ | OIDC | - | 🟢 | +------------------------+-------+--------+ | QuayDocumentation | - | 🟢 | +------------------------+-------+--------+ | Redis | - | 🟢 | +------------------------+-------+--------+ | RepoMirror | - | 🟢 | +------------------------+-------+--------+ | SecurityScanner | - | 🟢 | +------------------------+-------+--------+ | TeamSyncing | - | 🟢 | +------------------------+-------+--------+ | TimeMachine | - | 🟢 | +------------------------+-------+--------+ | UserVisibleSettings | - | 🟢 | +------------------------+-------+--------+ Running init script '/quay-registry/conf/init/nginx_conf_create.sh' Running init script '/quay-registry/conf/init/supervisord_conf_create.sh' Running init script '/quay-registry/conf/init/zz_boot.sh' /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) self._legacy_model = V2SecurityScanner(app, instance_keys, storage) Failed to validate security scanner V2 configuration /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) self._legacy_model = NoopV2SecurityScanner() 2022-05-05 10:00:39,221 INFO RPC interface 'supervisor' initialized 2022-05-05 10:00:39,221 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2022-05-05 10:00:39,221 INFO supervisord started with pid 7 2022-05-05 10:00:40,223 INFO spawned: 'stdout' with pid 54 2022-05-05 10:00:40,225 INFO spawned: 'blobuploadcleanupworker' with pid 55 2022-05-05 10:00:40,228 INFO spawned: 'builder' with pid 56 2022-05-05 10:00:40,230 INFO spawned: 'buildlogsarchiver' with pid 57 2022-05-05 10:00:40,232 INFO spawned: 'chunkcleanupworker' with pid 58 2022-05-05 10:00:40,234 INFO spawned: 'dnsmasq' with pid 59 2022-05-05 10:00:40,236 INFO spawned: 'expiredappspecifictokenworker' with pid 60 2022-05-05 10:00:40,238 INFO spawned: 'exportactionlogsworker' with pid 61 2022-05-05 10:00:40,241 INFO spawned: 'gcworker' with pid 62 2022-05-05 10:00:40,243 INFO spawned: 'globalpromstats' with pid 63 2022-05-05 10:00:40,246 INFO spawned: 'gunicorn-registry' with pid 64 2022-05-05 10:00:40,248 INFO spawned: 'gunicorn-secscan' with pid 65 2022-05-05 10:00:40,265 INFO spawned: 'gunicorn-web' with pid 66 2022-05-05 10:00:40,267 INFO spawned: 'jwtproxy' with pid 67 2022-05-05 10:00:40,269 INFO spawned: 'logrotateworker' with pid 68 2022-05-05 10:00:40,272 INFO spawned: 'manifestbackfillworker' with pid 69 2022-05-05 10:00:40,275 INFO spawned: 'memcache' with pid 70 2022-05-05 10:00:40,281 INFO spawned: 'namespacegcworker' with pid 71 2022-05-05 10:00:40,286 INFO spawned: 'nginx' with pid 75 2022-05-05 10:00:40,291 INFO spawned: 'notificationworker' with pid 76 2022-05-05 10:00:40,301 INFO spawned: 'pushgateway' with pid 77 2022-05-05 10:00:40,386 INFO spawned: 'queuecleanupworker' with pid 88 2022-05-05 10:00:40,401 INFO spawned: 'repositoryactioncounter' with pid 92 2022-05-05 10:00:40,403 INFO spawned: 'repositorygcworker' with pid 93 2022-05-05 10:00:40,404 INFO spawned: 'securityscanningnotificationworker' with pid 94 2022-05-05 10:00:40,477 INFO spawned: 'securityworker' with pid 97 2022-05-05 10:00:40,479 INFO spawned: 'servicekey' with pid 98 2022-05-05 10:00:40,487 INFO spawned: 'storagereplication' with pid 101 2022-05-05 10:00:40,571 INFO spawned: 'teamsyncworker' with pid 104 2022-05-05 10:00:41,688 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: blobuploadcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: builder entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: buildlogsarchiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: chunkcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: expiredappspecifictokenworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: exportactionlogsworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: gcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: globalpromstats entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: gunicorn-registry entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: gunicorn-secscan entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: gunicorn-web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: jwtproxy entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: logrotateworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,689 INFO success: manifestbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: memcache entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: namespacegcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: notificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: pushgateway entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: queuecleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: repositoryactioncounter entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: repositorygcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: securityscanningnotificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: securityworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: servicekey entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: storagereplication entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-05-05 10:00:41,690 INFO success: teamsyncworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) dnsmasq stderr | dnsmasq: started, version 2.79 cachesize 150 dnsmasq stderr | dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify dnsmasq stderr | dnsmasq: reading /etc/resolv.conf dnsmasq stderr | dnsmasq: using nameserver 172.30.0.10#53 dnsmasq stderr | dnsmasq: read /etc/hosts - 7 addresses jwtproxy stderr | time="2022-05-05T10:00:40Z" level=info msg="No claims verifiers specified, upstream should be configured to verify authorization" jwtproxy stderr | time="2022-05-05T10:00:40Z" level=info msg="Starting reverse proxy (Listening on 'unix:/tmp/jwtproxy_secscan.sock')" jwtproxy stderr | time="2022-05-05T10:00:40Z" level=info msg="Starting forward proxy (Listening on ':8081')" nginx stdout | 2022/05/05 10:00:40 [alert] 106#0: setpriority(-10) failed (13: Permission denied) pushgateway stderr | level=info ts=2022-05-05T10:00:40.489Z caller=main.go:85 msg="starting pushgateway" version="(version=, branch=, revision=)" pushgateway stderr | level=info ts=2022-05-05T10:00:40.490Z caller=main.go:86 build_context="(go=go1.15.14, user=, date=)" pushgateway stderr | level=info ts=2022-05-05T10:00:40.492Z caller=main.go:139 listen_address=:9091 pushgateway stderr | level=info ts=2022-05-05T10:00:40.493Z caller=tls_config.go:191 msg="TLS is disabled." http2=false nginx stdout | 2022/05/05 10:00:40 [alert] 107#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/05 10:00:40 [alert] 110#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/05 10:00:40 [alert] 111#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/05 10:00:40 [alert] 108#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/05 10:00:40 [alert] 105#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/05 10:00:40 [alert] 109#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/05 10:00:40 [alert] 112#0: setpriority(-10) failed (13: Permission denied) nginx stdout | 2022/05/05 10:00:47 [crit] 106#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.130.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.130.2.27:8080" nginx stdout | 2022/05/05 10:00:47 [crit] 106#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.130.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.130.2.27:8080" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:00:47 +0000] "GET /health/instance HTTP/1.1" 502 173 "-" "kube-probe/1.22" (0.000 118 0.000 : 0.000) gunicorn-secscan stdout | 2022-05-05 10:01:01,573 [65] [INFO] [data.registry_model] =============================== gunicorn-secscan stdout | 2022-05-05 10:01:01,577 [65] [INFO] [data.registry_model] Using registry model `` gunicorn-secscan stdout | 2022-05-05 10:01:01,581 [65] [INFO] [data.registry_model] =============================== nginx stdout | 2022/05/05 10:01:02 [crit] 106#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.130.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.130.2.27:8080" nginx stdout | 2022/05/05 10:01:02 [crit] 106#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.130.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.130.2.27:8080" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:01:02 +0000] "GET /health/instance HTTP/1.1" 502 173 "-" "kube-probe/1.22" (0.000 118 0.000 : 0.000) expiredappspecifictokenworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) expiredappspecifictokenworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) expiredappspecifictokenworker stderr | Failed to validate security scanner V2 configuration expiredappspecifictokenworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) expiredappspecifictokenworker stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-secscan stdout | 2022-05-05 10:01:02,686 [65] [DEBUG] [app] Loading default config. gunicorn-secscan stdout | 2022-05-05 10:01:02,686 [65] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-secscan stdout | 2022-05-05 10:01:02,767 [65] [DEBUG] [app] Loaded config gunicorn-secscan stdout | 2022-05-05 10:01:02,768 [65] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-secscan stdout | 2022-05-05 10:01:02,787 [65] [DEBUG] [util.ipresolver] Building AWS IP ranges expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,171 [60] [DEBUG] [__main__] Starting expired app specific token GC worker expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,379 [60] [DEBUG] [__main__] Found expiration window: 1d gunicorn-web stdout | 2022-05-05 10:01:03,380 [66] [INFO] [data.registry_model] =============================== gunicorn-web stdout | 2022-05-05 10:01:03,380 [66] [INFO] [data.registry_model] Using registry model `` gunicorn-web stdout | 2022-05-05 10:01:03,380 [66] [INFO] [data.registry_model] =============================== expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,384 [60] [DEBUG] [workers.worker] Scheduling worker. expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,386 [60] [INFO] [apscheduler.scheduler] Scheduler started expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,479 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,480 [60] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:02:15.480971 expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,481 [60] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,872 [60] [INFO] [apscheduler.scheduler] Added job "ExpiredAppSpecificTokenWorker._gc_expired_tokens" to job store "default" expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,875 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2022-05-05 10:01:03,875 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:15.480971+00:00 (in 71.605052 seconds) gunicorn-web stdout | 2022-05-05 10:01:05,698 [66] [DEBUG] [app] Loading default config. gunicorn-web stdout | 2022-05-05 10:01:05,702 [66] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-web stdout | 2022-05-05 10:01:05,770 [66] [DEBUG] [app] Loaded config gunicorn-web stdout | 2022-05-05 10:01:05,778 [66] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-web stdout | 2022-05-05 10:01:05,888 [66] [DEBUG] [util.ipresolver] Building AWS IP ranges storagereplication stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) storagereplication stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) storagereplication stderr | Failed to validate security scanner V2 configuration storagereplication stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) storagereplication stderr | self._legacy_model = NoopV2SecurityScanner() storagereplication stdout | 2022-05-05 10:01:09,873 [101] [DEBUG] [__main__] Full storage replication disabled; skipping blobuploadcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) blobuploadcleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) blobuploadcleanupworker stderr | Failed to validate security scanner V2 configuration blobuploadcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) blobuploadcleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() repositorygcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositorygcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) repositorygcworker stderr | Failed to validate security scanner V2 configuration repositorygcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositorygcworker stderr | self._legacy_model = NoopV2SecurityScanner() blobuploadcleanupworker stdout | 2022-05-05 10:01:10,984 [55] [DEBUG] [workers.worker] Scheduling worker. blobuploadcleanupworker stdout | 2022-05-05 10:01:10,987 [55] [INFO] [apscheduler.scheduler] Scheduler started blobuploadcleanupworker stdout | 2022-05-05 10:01:11,067 [55] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2022-05-05 10:01:11,067 [55] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:11:04.068940 blobuploadcleanupworker stdout | 2022-05-05 10:01:11,070 [55] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added blobuploadcleanupworker stdout | 2022-05-05 10:01:11,387 [55] [INFO] [apscheduler.scheduler] Added job "BlobUploadCleanupWorker._try_cleanup_uploads" to job store "default" blobuploadcleanupworker stdout | 2022-05-05 10:01:11,463 [55] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2022-05-05 10:01:11,463 [55] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:04.068940+00:00 (in 592.605023 seconds) repositorygcworker stdout | 2022-05-05 10:01:11,579 [93] [DEBUG] [__main__] Starting repository GC worker repositorygcworker stdout | 2022-05-05 10:01:11,881 [93] [DEBUG] [workers.worker] Scheduling worker. repositorygcworker stdout | 2022-05-05 10:01:11,883 [93] [INFO] [apscheduler.scheduler] Scheduler started repositorygcworker stdout | 2022-05-05 10:01:11,983 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:01:11,984 [93] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:24.984990 repositorygcworker stdout | 2022-05-05 10:01:11,985 [93] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositorygcworker stdout | 2022-05-05 10:01:12,379 [93] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" repositorygcworker stdout | 2022-05-05 10:01:12,384 [93] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:03:00.385766 repositorygcworker stdout | 2022-05-05 10:01:12,384 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:01:12,386 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:24.984990+00:00 (in 12.598937 seconds) repositorygcworker stdout | 2022-05-05 10:01:12,392 [93] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" repositorygcworker stdout | 2022-05-05 10:01:12,393 [93] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:13.394619 repositorygcworker stdout | 2022-05-05 10:01:12,393 [93] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" repositorygcworker stdout | 2022-05-05 10:01:12,463 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:01:12,464 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:13.394619+00:00 (in 0.930609 seconds) gunicorn-secscan stdout | 2022-05-05 10:01:13,285 [65] [DEBUG] [util.ipresolver] Finished building AWS IP ranges servicekey stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) servicekey stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) servicekey stderr | Failed to validate security scanner V2 configuration servicekey stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) servicekey stderr | self._legacy_model = NoopV2SecurityScanner() repositorygcworker stdout | 2022-05-05 10:01:13,463 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:01:13,473 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:01:13 UTC)" (scheduled at 2022-05-05 10:01:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:01:13,474 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:24.984990+00:00 (in 11.520787 seconds) repositorygcworker stdout | 2022-05-05 10:01:13,474 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:01:13,474 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:02:13 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:01:13,767 [65] [DEBUG] [data.database] Configuring database gunicorn-secscan stdout | 2022-05-05 10:01:13,770 [65] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-secscan stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-secscan stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-secscan stdout | 2022-05-05 10:01:13,773 [65] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-secscan stdout | 2022-05-05 10:01:13,774 [65] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-secscan stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-secscan stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-secscan stdout | 2022-05-05 10:01:13,776 [65] [INFO] [data.secscan_model] =============================== gunicorn-secscan stdout | 2022-05-05 10:01:13,777 [65] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-secscan stdout | 2022-05-05 10:01:13,778 [65] [INFO] [data.secscan_model] =============================== gunicorn-secscan stdout | 2022-05-05 10:01:13,779 [65] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-secscan stdout | 2022-05-05 10:01:13,780 [65] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2022-05-05 10:01:13,781 [65] [INFO] [data.logs_model] Using logs model `` gunicorn-secscan stdout | 2022-05-05 10:01:13,782 [65] [INFO] [data.logs_model] =============================== servicekey stdout | 2022-05-05 10:01:13,864 [98] [DEBUG] [workers.worker] Scheduling worker. servicekey stdout | 2022-05-05 10:01:13,865 [98] [INFO] [apscheduler.scheduler] Scheduler started servicekey stdout | 2022-05-05 10:01:13,867 [98] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2022-05-05 10:01:13,867 [98] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:27:11.868820 gunicorn-secscan stdout | 2022-05-05 10:01:13,870 [65] [DEBUG] [__config__] Starting secscan gunicorn with 2 workers and gevent worker class servicekey stdout | 2022-05-05 10:01:13,868 [98] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added servicekey stdout | 2022-05-05 10:01:14,065 [98] [INFO] [apscheduler.scheduler] Added job "ServiceKeyWorker._refresh_service_key" to job store "default" servicekey stdout | 2022-05-05 10:01:14,065 [98] [DEBUG] [apscheduler.scheduler] Looking for jobs to run servicekey stdout | 2022-05-05 10:01:14,065 [98] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:27:11.868820+00:00 (in 1557.802891 seconds) chunkcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) chunkcleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gcworker stderr | Failed to validate security scanner V2 configuration gcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gcworker stderr | self._legacy_model = NoopV2SecurityScanner() chunkcleanupworker stderr | Failed to validate security scanner V2 configuration chunkcleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) chunkcleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() namespacegcworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) namespacegcworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) namespacegcworker stderr | Failed to validate security scanner V2 configuration namespacegcworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) namespacegcworker stderr | self._legacy_model = NoopV2SecurityScanner() exportactionlogsworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) exportactionlogsworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) exportactionlogsworker stderr | Failed to validate security scanner V2 configuration exportactionlogsworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) exportactionlogsworker stderr | self._legacy_model = NoopV2SecurityScanner() namespacegcworker stdout | 2022-05-05 10:01:14,767 [71] [DEBUG] [__main__] Starting namespace GC worker gcworker stdout | 2022-05-05 10:01:14,780 [62] [DEBUG] [workers.worker] Scheduling worker. gcworker stdout | 2022-05-05 10:01:14,783 [62] [INFO] [apscheduler.scheduler] Scheduler started chunkcleanupworker stdout | 2022-05-05 10:01:14,789 [58] [DEBUG] [__main__] Swift storage not detected; sleeping gcworker stdout | 2022-05-05 10:01:14,863 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:01:14,864 [62] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:26.865082 gcworker stdout | 2022-05-05 10:01:14,864 [62] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added namespacegcworker stdout | 2022-05-05 10:01:14,883 [71] [DEBUG] [workers.worker] Scheduling worker. namespacegcworker stdout | 2022-05-05 10:01:14,884 [71] [INFO] [apscheduler.scheduler] Scheduler started namespacegcworker stdout | 2022-05-05 10:01:14,976 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:01:14,978 [71] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added namespacegcworker stdout | 2022-05-05 10:01:14,977 [71] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:38.977973 gcworker stdout | 2022-05-05 10:01:15,068 [62] [INFO] [apscheduler.scheduler] Added job "GarbageCollectionWorker._garbage_collection_repos" to job store "default" gcworker stdout | 2022-05-05 10:01:15,068 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:01:15,068 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:26.865082+00:00 (in 11.796165 seconds) namespacegcworker stdout | 2022-05-05 10:01:15,278 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" namespacegcworker stdout | 2022-05-05 10:01:15,280 [71] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:04:23.280939 namespacegcworker stdout | 2022-05-05 10:01:15,280 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:01:15,282 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" namespacegcworker stdout | 2022-05-05 10:01:15,285 [71] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:02:14.286471 namespacegcworker stdout | 2022-05-05 10:01:15,285 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:38.977973+00:00 (in 23.695616 seconds) namespacegcworker stdout | 2022-05-05 10:01:15,285 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:01:15,285 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:38.977973+00:00 (in 23.692028 seconds) namespacegcworker stdout | 2022-05-05 10:01:15,286 [71] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" namespacegcworker stdout | 2022-05-05 10:01:15,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:01:15,286 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:38.977973+00:00 (in 23.691689 seconds) builder stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) builder stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) builder stderr | Failed to validate security scanner V2 configuration builder stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) builder stderr | self._legacy_model = NoopV2SecurityScanner() securityscanningnotificationworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityscanningnotificationworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) securityscanningnotificationworker stderr | Failed to validate security scanner V2 configuration securityscanningnotificationworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityscanningnotificationworker stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-web stdout | 2022-05-05 10:01:16,571 [66] [DEBUG] [util.ipresolver] Finished building AWS IP ranges securityscanningnotificationworker stdout | 2022-05-05 10:01:16,665 [94] [DEBUG] [__main__] Starting security scanning notification worker securityscanningnotificationworker stdout | 2022-05-05 10:01:16,768 [94] [DEBUG] [workers.worker] Scheduling worker. securityscanningnotificationworker stdout | 2022-05-05 10:01:16,771 [94] [INFO] [apscheduler.scheduler] Scheduler started securityscanningnotificationworker stdout | 2022-05-05 10:01:16,772 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:01:16,773 [94] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:31.774027 securityscanningnotificationworker stdout | 2022-05-05 10:01:16,774 [94] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityscanningnotificationworker stdout | 2022-05-05 10:01:16,907 [94] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" securityscanningnotificationworker stdout | 2022-05-05 10:01:16,964 [94] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:03:03.965762 securityscanningnotificationworker stdout | 2022-05-05 10:01:16,964 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:01:16,966 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:31.774027+00:00 (in 14.807854 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:01:16,967 [94] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" securityscanningnotificationworker stdout | 2022-05-05 10:01:16,968 [94] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:35.969483 securityscanningnotificationworker stdout | 2022-05-05 10:01:16,968 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:01:16,969 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:31.774027+00:00 (in 14.804388 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:01:16,971 [94] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" securityscanningnotificationworker stdout | 2022-05-05 10:01:16,972 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:01:16,972 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:31.774027+00:00 (in 14.801807 seconds) gunicorn-web stdout | 2022-05-05 10:01:17,082 [66] [DEBUG] [data.database] Configuring database gunicorn-web stdout | 2022-05-05 10:01:17,084 [66] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-web stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-web stdout | 2022-05-05 10:01:17,091 [66] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-web stdout | 2022-05-05 10:01:17,093 [66] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-web stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-web stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-web stdout | 2022-05-05 10:01:17,099 [66] [INFO] [data.secscan_model] =============================== gunicorn-web stdout | 2022-05-05 10:01:17,100 [66] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-web stdout | 2022-05-05 10:01:17,165 [66] [INFO] [data.secscan_model] =============================== gunicorn-web stdout | 2022-05-05 10:01:17,166 [66] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-web stdout | 2022-05-05 10:01:17,167 [66] [INFO] [data.logs_model] =============================== gunicorn-web stdout | 2022-05-05 10:01:17,168 [66] [INFO] [data.logs_model] Using logs model `` gunicorn-web stdout | 2022-05-05 10:01:17,169 [66] [INFO] [data.logs_model] =============================== repositoryactioncounter stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositoryactioncounter stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) repositoryactioncounter stderr | Failed to validate security scanner V2 configuration repositoryactioncounter stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) repositoryactioncounter stderr | self._legacy_model = NoopV2SecurityScanner() nginx stdout | 2022/05/05 10:01:17 [crit] 106#0: *7 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.130.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.130.2.27:8080" nginx stdout | 2022/05/05 10:01:17 [crit] 106#0: *7 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.130.2.1, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.130.2.27:8080" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:01:17 +0000] "GET /health/instance HTTP/1.1" 502 173 "-" "kube-probe/1.22" (0.000 118 0.000 : 0.000) globalpromstats stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) globalpromstats stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) globalpromstats stderr | Failed to validate security scanner V2 configuration globalpromstats stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) globalpromstats stderr | self._legacy_model = NoopV2SecurityScanner() repositoryactioncounter stdout | 2022-05-05 10:01:17,673 [92] [DEBUG] [workers.worker] Scheduling worker. repositoryactioncounter stdout | 2022-05-05 10:01:17,674 [92] [INFO] [apscheduler.scheduler] Scheduler started builder stdout | 2022-05-05 10:01:17,688 [56] [DEBUG] [__main__] Building is disabled. Please enable the feature flag repositoryactioncounter stdout | 2022-05-05 10:01:17,763 [92] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2022-05-05 10:01:17,764 [92] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 11:59:33.765018 repositoryactioncounter stdout | 2022-05-05 10:01:17,764 [92] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added buildlogsarchiver stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) buildlogsarchiver stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) buildlogsarchiver stderr | Failed to validate security scanner V2 configuration buildlogsarchiver stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) buildlogsarchiver stderr | self._legacy_model = NoopV2SecurityScanner() exportactionlogsworker stdout | 2022-05-05 10:01:17,789 [61] [DEBUG] [__main__] Starting export action logs worker teamsyncworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) teamsyncworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) teamsyncworker stderr | Failed to validate security scanner V2 configuration teamsyncworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) teamsyncworker stderr | self._legacy_model = NoopV2SecurityScanner() exportactionlogsworker stdout | 2022-05-05 10:01:17,969 [61] [DEBUG] [workers.worker] Scheduling worker. exportactionlogsworker stdout | 2022-05-05 10:01:17,971 [61] [INFO] [apscheduler.scheduler] Scheduler started exportactionlogsworker stdout | 2022-05-05 10:01:17,977 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:01:17,977 [61] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:02:00.978909 exportactionlogsworker stdout | 2022-05-05 10:01:17,979 [61] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added repositoryactioncounter stdout | 2022-05-05 10:01:18,074 [92] [INFO] [apscheduler.scheduler] Added job "RepositoryActionCountWorker._run_counting" to job store "default" repositoryactioncounter stdout | 2022-05-05 10:01:18,076 [92] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositoryactioncounter stdout | 2022-05-05 10:01:18,076 [92] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 11:59:33.765018+00:00 (in 7095.688263 seconds) globalpromstats stdout | 2022-05-05 10:01:18,170 [63] [DEBUG] [workers.worker] Scheduling worker. manifestbackfillworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) manifestbackfillworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) manifestbackfillworker stderr | Failed to validate security scanner V2 configuration manifestbackfillworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) manifestbackfillworker stderr | self._legacy_model = NoopV2SecurityScanner() exportactionlogsworker stdout | 2022-05-05 10:01:18,181 [61] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" exportactionlogsworker stdout | 2022-05-05 10:01:18,183 [61] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:05:59.184038 exportactionlogsworker stdout | 2022-05-05 10:01:18,183 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:01:18,184 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:00.978909+00:00 (in 42.794780 seconds) exportactionlogsworker stdout | 2022-05-05 10:01:18,185 [61] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" globalpromstats stdout | 2022-05-05 10:01:18,185 [63] [INFO] [apscheduler.scheduler] Scheduler started exportactionlogsworker stdout | 2022-05-05 10:01:18,186 [61] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:47.187465 exportactionlogsworker stdout | 2022-05-05 10:01:18,186 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:01:18,187 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:00.978909+00:00 (in 42.791385 seconds) exportactionlogsworker stdout | 2022-05-05 10:01:18,188 [61] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" exportactionlogsworker stdout | 2022-05-05 10:01:18,263 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:01:18,263 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:47.187465+00:00 (in 28.923560 seconds) globalpromstats stdout | 2022-05-05 10:01:18,276 [63] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2022-05-05 10:01:18,277 [63] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:54:32.277998 globalpromstats stdout | 2022-05-05 10:01:18,278 [63] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added teamsyncworker stdout | 2022-05-05 10:01:18,376 [104] [DEBUG] [__main__] Team syncing is disabled; sleeping buildlogsarchiver stdout | 2022-05-05 10:01:18,476 [57] [DEBUG] [workers.worker] Scheduling worker. buildlogsarchiver stdout | 2022-05-05 10:01:18,477 [57] [INFO] [apscheduler.scheduler] Scheduler started globalpromstats stdout | 2022-05-05 10:01:18,479 [63] [INFO] [apscheduler.scheduler] Added job "GlobalPrometheusStatsWorker._try_report_stats" to job store "default" buildlogsarchiver stdout | 2022-05-05 10:01:18,478 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:01:18,479 [57] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:44.479974 globalpromstats stdout | 2022-05-05 10:01:18,481 [63] [DEBUG] [apscheduler.scheduler] Looking for jobs to run globalpromstats stdout | 2022-05-05 10:01:18,481 [63] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:54:32.277998+00:00 (in 3193.796075 seconds) buildlogsarchiver stdout | 2022-05-05 10:01:18,479 [57] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added manifestbackfillworker stdout | 2022-05-05 10:01:18,667 [69] [DEBUG] [workers.worker] Scheduling worker. manifestbackfillworker stdout | 2022-05-05 10:01:18,669 [69] [INFO] [apscheduler.scheduler] Scheduler started notificationworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) notificationworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) notificationworker stderr | Failed to validate security scanner V2 configuration notificationworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) notificationworker stderr | self._legacy_model = NoopV2SecurityScanner() manifestbackfillworker stdout | 2022-05-05 10:01:18,677 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2022-05-05 10:01:18,681 [69] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added manifestbackfillworker stdout | 2022-05-05 10:01:18,678 [69] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:57:33.679019 securityworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) securityworker stderr | Failed to validate security scanner V2 configuration securityworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) securityworker stderr | self._legacy_model = NoopV2SecurityScanner() logrotateworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) logrotateworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) logrotateworker stderr | Failed to validate security scanner V2 configuration logrotateworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) logrotateworker stderr | self._legacy_model = NoopV2SecurityScanner() buildlogsarchiver stdout | 2022-05-05 10:01:18,794 [57] [INFO] [apscheduler.scheduler] Added job "ArchiveBuildLogsWorker._archive_redis_buildlogs" to job store "default" buildlogsarchiver stdout | 2022-05-05 10:01:18,795 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:01:18,795 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:44.479974+00:00 (in 25.684353 seconds) manifestbackfillworker stdout | 2022-05-05 10:01:18,891 [69] [INFO] [apscheduler.scheduler] Added job "ManifestBackfillWorker._backfill_manifests" to job store "default" manifestbackfillworker stdout | 2022-05-05 10:01:18,893 [69] [DEBUG] [apscheduler.scheduler] Looking for jobs to run manifestbackfillworker stdout | 2022-05-05 10:01:18,894 [69] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:57:33.679019+00:00 (in 3374.784787 seconds) queuecleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) queuecleanupworker stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) queuecleanupworker stderr | Failed to validate security scanner V2 configuration queuecleanupworker stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) queuecleanupworker stderr | self._legacy_model = NoopV2SecurityScanner() notificationworker stdout | 2022-05-05 10:01:19,077 [76] [DEBUG] [workers.worker] Scheduling worker. notificationworker stdout | 2022-05-05 10:01:19,078 [76] [INFO] [apscheduler.scheduler] Scheduler started notificationworker stdout | 2022-05-05 10:01:19,088 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:01:19,089 [76] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:23.089985 notificationworker stdout | 2022-05-05 10:01:19,089 [76] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added logrotateworker stdout | 2022-05-05 10:01:19,173 [68] [DEBUG] [__main__] Action log rotation worker not enabled; skipping securityworker stdout | 2022-05-05 10:01:19,198 [97] [DEBUG] [workers.worker] Scheduling worker. securityworker stdout | 2022-05-05 10:01:19,199 [97] [INFO] [apscheduler.scheduler] Scheduler started securityworker stdout | 2022-05-05 10:01:19,200 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:01:19,200 [97] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:01:28.201650 notificationworker stdout | 2022-05-05 10:01:19,205 [76] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" notificationworker stdout | 2022-05-05 10:01:19,206 [76] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:05:32.207092 notificationworker stdout | 2022-05-05 10:01:19,206 [76] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" notificationworker stdout | 2022-05-05 10:01:19,207 [76] [DEBUG] [workers.worker] First run scheduled for 2022-05-05 10:02:12.207990 notificationworker stdout | 2022-05-05 10:01:19,207 [76] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" notificationworker stdout | 2022-05-05 10:01:19,207 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:01:19,207 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:23.089985+00:00 (in 3.882491 seconds) securityworker stdout | 2022-05-05 10:01:19,201 [97] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added securityworker stdout | 2022-05-05 10:01:19,288 [97] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_in_scanner" to job store "default" securityworker stdout | 2022-05-05 10:01:19,288 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:01:19,288 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:28.201650+00:00 (in 8.913162 seconds) queuecleanupworker stdout | 2022-05-05 10:01:19,293 [88] [DEBUG] [workers.worker] Scheduling worker. queuecleanupworker stdout | 2022-05-05 10:01:19,294 [88] [INFO] [apscheduler.scheduler] Scheduler started queuecleanupworker stdout | 2022-05-05 10:01:19,294 [88] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2022-05-05 10:01:19,294 [88] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added queuecleanupworker stdout | 2022-05-05 10:01:19,295 [88] [DEBUG] [workers.worker] First run scheduled for 2022-05-06 04:36:38.296056 queuecleanupworker stdout | 2022-05-05 10:01:19,380 [88] [INFO] [apscheduler.scheduler] Added job "QueueCleanupWorker._cleanup_queue" to job store "default" queuecleanupworker stdout | 2022-05-05 10:01:19,381 [88] [DEBUG] [apscheduler.scheduler] Looking for jobs to run queuecleanupworker stdout | 2022-05-05 10:01:19,381 [88] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-06 04:36:38.296056+00:00 (in 66918.914591 seconds) gunicorn-web stdout | 2022-05-05 10:01:19,714 [66] [DEBUG] [__config__] Starting web gunicorn with 4 workers and gevent worker class gunicorn-registry stdout | 2022-05-05 10:01:20,480 [64] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2022-05-05 10:01:20,480 [64] [INFO] [data.registry_model] Using registry model `` gunicorn-registry stdout | 2022-05-05 10:01:20,480 [64] [INFO] [data.registry_model] =============================== gunicorn-registry stdout | 2022-05-05 10:01:20,633 [64] [DEBUG] [app] Loading default config. gunicorn-registry stdout | 2022-05-05 10:01:20,633 [64] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml gunicorn-registry stdout | 2022-05-05 10:01:20,639 [64] [DEBUG] [app] Loaded config gunicorn-registry stdout | 2022-05-05 10:01:20,640 [64] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-registry stdout | 2022-05-05 10:01:20,650 [64] [DEBUG] [util.ipresolver] Building AWS IP ranges gunicorn-registry stdout | 2022-05-05 10:01:21,346 [64] [DEBUG] [util.ipresolver] Finished building AWS IP ranges gunicorn-registry stdout | 2022-05-05 10:01:21,436 [64] [DEBUG] [data.database] Configuring database gunicorn-registry stdout | 2022-05-05 10:01:21,437 [64] [INFO] [data.database] Connection pooling enabled for postgresql; stale timeout: None; max connection count: None gunicorn-registry stderr | /quay-registry/data/secscan_model/__init__.py:28: DeprecationWarning: Call to deprecated class V2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-registry stderr | self._legacy_model = V2SecurityScanner(app, instance_keys, storage) gunicorn-registry stdout | 2022-05-05 10:01:21,438 [64] [DEBUG] [util.secscan.validator] Missing SECURITY_SCANNER_ENDPOINT configuration gunicorn-registry stdout | 2022-05-05 10:01:21,438 [64] [WARNING] [data.secscan_model.secscan_v2_model] Failed to validate security scanner V2 configuration gunicorn-registry stdout | 2022-05-05 10:01:21,439 [64] [INFO] [data.secscan_model] =============================== gunicorn-registry stderr | /quay-registry/data/secscan_model/__init__.py:30: DeprecationWarning: Call to deprecated class NoopV2SecurityScanner. (Will be replaced by a V4 API security scanner soon) gunicorn-registry stderr | self._legacy_model = NoopV2SecurityScanner() gunicorn-registry stdout | 2022-05-05 10:01:21,439 [64] [INFO] [data.secscan_model] Using split secscan model: `[, ]` gunicorn-registry stdout | 2022-05-05 10:01:21,439 [64] [INFO] [data.secscan_model] =============================== gunicorn-registry stdout | 2022-05-05 10:01:21,439 [64] [DEBUG] [data.logs_model] Configuring log lodel gunicorn-registry stdout | 2022-05-05 10:01:21,439 [64] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2022-05-05 10:01:21,439 [64] [INFO] [data.logs_model] Using logs model `` gunicorn-registry stdout | 2022-05-05 10:01:21,439 [64] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2022-05-05 10:01:21,570 [64] [DEBUG] [__config__] Starting registry gunicorn with 8 workers and gevent worker class notificationworker stdout | 2022-05-05 10:01:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:01:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:01:23 UTC)" (scheduled at 2022-05-05 10:01:23.089985+00:00) notificationworker stdout | 2022-05-05 10:01:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:01:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 1, 23, 90907), True, datetime.datetime(2022, 5, 5, 10, 1, 23, 90907), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:01:23,092 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:33.089985+00:00 (in 9.999632 seconds) notificationworker stdout | 2022-05-05 10:01:23,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:01:23,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:01:23,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:01:33 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:01:24,815 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} repositorygcworker stdout | 2022-05-05 10:01:24,995 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:01:24,995 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:13.394619+00:00 (in 48.399170 seconds) repositorygcworker stdout | 2022-05-05 10:01:24,995 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:02:24 UTC)" (scheduled at 2022-05-05 10:01:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:01:24,996 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:01:24,998 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 1, 24, 996206), True, datetime.datetime(2022, 5, 5, 10, 1, 24, 996206), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:01:25,012 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:01:25,012 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:01:25,012 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:02:24 UTC)" executed successfully expiredappspecifictokenworker stdout | 2022-05-05 10:01:25,404 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:01:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:01:26,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:01:26 UTC)" (scheduled at 2022-05-05 10:01:26.865082+00:00) gcworker stdout | 2022-05-05 10:01:26,867 [62] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2022-05-05 10:01:26,868 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:56.865082+00:00 (in 29.999575 seconds) gcworker stdout | 2022-05-05 10:01:26,880 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535286879, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:01:26,885 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:01:26,885 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:01:26,885 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:01:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:01:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:01:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:01:28 UTC)" (scheduled at 2022-05-05 10:01:28.201650+00:00) securityworker stdout | 2022-05-05 10:01:28,203 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:01:28,203 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:01:28,204 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:58.201650+00:00 (in 29.999358 seconds) securityworker stdout | 2022-05-05 10:01:28,207 [97] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): quay-clair-app:80 securityworker stdout | 2022-05-05 10:01:28,214 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:01:28,219 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:01:28,230 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:01:28,234 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:01:28,237 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:01:28,237 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:01:28,238 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 9, 56, 28, 237600), 1, 2]) securityworker stdout | 2022-05-05 10:01:28,240 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:01:28,240 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:01:28,242 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 9, 56, 28, 240827), 1, 2]) securityworker stdout | 2022-05-05 10:01:28,244 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:01:28,244 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:01:28,245 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:01:28,245 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:01:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:01:29,203 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:01:29,404 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:01:29,708 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:01:30,305 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:01:30,620 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:01:31,020 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:01:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:01:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:01:31 UTC)" (scheduled at 2022-05-05 10:01:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:01:31,775 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:35.969483+00:00 (in 4.195035 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:01:31,776 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:01:31,779 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 1, 31, 777185), True, datetime.datetime(2022, 5, 5, 10, 1, 31, 777185), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:01:31,790 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:01:31,790 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:01:31,791 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:02:31 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-05 10:01:32,312 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} gunicorn-web stdout | 2022-05-05 10:01:32,375 [220] [DEBUG] [app] Starting request: urn:request:4b15dc5a-3127-41b3-b1e9-34393988c2cb (/health/instance) gunicorn-web stdout | 2022-05-05 10:01:32,386 [220] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 gunicorn-registry stdout | 2022-05-05 10:01:32,401 [227] [DEBUG] [app] Starting request: urn:request:f3ee216e-40e2-426e-b55b-3c70d0bed123 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:01:32,405 [227] [DEBUG] [app] Ending request: urn:request:f3ee216e-40e2-426e-b55b-3c70d0bed123 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:01:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.009) gunicorn-registry stdout | 2022-05-05 10:01:32,406 [227] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:01:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:01:32,406 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:01:32,409 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:01:32,412 [220] [DEBUG] [app] Starting request: urn:request:8836f1e7-9f74-4a2f-9577-9a317f988a9e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:01:32,416 [220] [DEBUG] [app] Ending request: urn:request:8836f1e7-9f74-4a2f-9577-9a317f988a9e (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:01:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 159 0.006) gunicorn-web stdout | 2022-05-05 10:01:32,417 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:01:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:01:32,418 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:01:32,422 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 1, 32, 420419), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 1, 32, 420464), 'quay', None, datetime.datetime(2022, 5, 4, 10, 1, 32, 420485), datetime.datetime(2022, 4, 28, 10, 1, 32, 420507), None]) builder stdout | 2022-05-05 10:01:32,422 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-web stdout | 2022-05-05 10:01:32,437 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:01:32,444 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:01:32,447 [220] [DEBUG] [app] Ending request: urn:request:4b15dc5a-3127-41b3-b1e9-34393988c2cb (/health/instance) gunicorn-web stdout | 2022-05-05 10:01:32,447 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:01:32,448 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:01:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:01:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.080 118 0.079) gunicorn-web stdout | 2022-05-05 10:01:32,451 [220] [DEBUG] [app] Starting request: urn:request:450efa0b-4b59-4b29-a231-c528bb997e27 (/health/instance) gunicorn-registry stdout | 2022-05-05 10:01:32,458 [231] [DEBUG] [app] Starting request: urn:request:84253ba6-cf6a-4eb9-b91d-9ce0f04472f8 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:01:32,462 [231] [DEBUG] [app] Ending request: urn:request:84253ba6-cf6a-4eb9-b91d-9ce0f04472f8 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:01:32,464 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:01:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:01:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.014 162 0.014) gunicorn-web stdout | 2022-05-05 10:01:32,468 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:01:32,471 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:01:32,478 [222] [DEBUG] [app] Starting request: urn:request:3ed75891-359a-4859-b7d9-a1b2331c5254 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:01:32,480 [222] [DEBUG] [app] Ending request: urn:request:3ed75891-359a-4859-b7d9-a1b2331c5254 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:01:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 159 0.009) gunicorn-web stdout | 2022-05-05 10:01:32,482 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:01:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:01:32,482 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:01:32,483 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:01:32,491 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:01:32,501 [220] [DEBUG] [app] Ending request: urn:request:450efa0b-4b59-4b29-a231-c528bb997e27 (/health/instance) gunicorn-web stdout | 2022-05-05 10:01:32,502 [220] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:01:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.054 118 0.054) gunicorn-web stdout | 2022-05-05 10:01:32,505 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:01:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-secscan stdout | 2022-05-05 10:01:32,793 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:01:32,794 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:01:32,796 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} buildlogsarchiver stdout | 2022-05-05 10:01:32,905 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:01:32,928 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} notificationworker stdout | 2022-05-05 10:01:33,092 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:01:33,092 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:43.089985+00:00 (in 9.997420 seconds) notificationworker stdout | 2022-05-05 10:01:33,093 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:01:43 UTC)" (scheduled at 2022-05-05 10:01:33.089985+00:00) notificationworker stdout | 2022-05-05 10:01:33,093 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:01:33,095 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 1, 33, 93535), True, datetime.datetime(2022, 5, 5, 10, 1, 33, 93535), 0, 'notification/%', 50, 1, 0]) teamsyncworker stdout | 2022-05-05 10:01:33,106 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} notificationworker stdout | 2022-05-05 10:01:33,107 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:01:33,108 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:01:33,108 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:01:43 UTC)" executed successfully repositoryactioncounter stdout | 2022-05-05 10:01:33,817 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:01:34,810 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:01:35,013 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:01:35,102 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:01:35,328 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:01:35,613 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:01:35,794 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:01:35,803 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:01:35,808 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:01:35,811 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:01:35,815 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:01:35,972 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:01:35,972 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:31.774027+00:00 (in 55.801578 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:01:35,972 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:02:35 UTC)" (scheduled at 2022-05-05 10:01:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:01:35,973 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:01:35,973 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:02:35 UTC)" executed successfully queuecleanupworker stdout | 2022-05-05 10:01:36,512 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:01:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:01:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:01:38 UTC)" (scheduled at 2022-05-05 10:01:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:01:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:01:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 1, 38, 978843), True, datetime.datetime(2022, 5, 5, 10, 1, 38, 978843), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:01:38,980 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:14.286471+00:00 (in 35.308172 seconds) namespacegcworker stdout | 2022-05-05 10:01:38,989 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:01:38,989 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:01:38,990 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:02:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:01:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:01:43,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:01:53.089985+00:00 (in 9.999085 seconds) notificationworker stdout | 2022-05-05 10:01:43,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:01:53 UTC)" (scheduled at 2022-05-05 10:01:43.089985+00:00) notificationworker stdout | 2022-05-05 10:01:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:01:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 1, 43, 91379), True, datetime.datetime(2022, 5, 5, 10, 1, 43, 91379), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:01:43,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:01:43,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:01:43,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:01:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:01:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:01:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:01:44 UTC)" (scheduled at 2022-05-05 10:01:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:01:44,481 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:14.479974+00:00 (in 29.999527 seconds) buildlogsarchiver stdout | 2022-05-05 10:01:44,482 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 1, 44, 481220), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:01:44,498 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:01:44,498 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:01:44,498 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:01:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:01:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:01:47 UTC)" (scheduled at 2022-05-05 10:01:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:01:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:01:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:01:47 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:01:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:00.978909+00:00 (in 13.791040 seconds) gunicorn-web stdout | 2022-05-05 10:01:47,369 [222] [DEBUG] [app] Starting request: urn:request:ba5f16aa-8a2e-4800-86e7-b2ec9ff62a9b (/health/instance) gunicorn-web stdout | 2022-05-05 10:01:47,372 [221] [DEBUG] [app] Starting request: urn:request:0cdf72c0-f01a-4dd0-be78-39d1bd7a37a2 (/health/instance) gunicorn-web stdout | 2022-05-05 10:01:47,377 [222] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 gunicorn-web stdout | 2022-05-05 10:01:47,379 [221] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 gunicorn-registry stdout | 2022-05-05 10:01:47,381 [231] [DEBUG] [app] Starting request: urn:request:c9e37d64-098a-4866-8677-8ac0e6ba9452 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:01:47,382 [231] [DEBUG] [app] Ending request: urn:request:c9e37d64-098a-4866-8677-8ac0e6ba9452 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:01:47,383 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:01:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:01:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-web stdout | 2022-05-05 10:01:47,383 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:01:47,383 [224] [DEBUG] [app] Starting request: urn:request:4f5d7986-7945-4edd-9e60-811b8e2766f6 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:01:47,385 [224] [DEBUG] [app] Ending request: urn:request:4f5d7986-7945-4edd-9e60-811b8e2766f6 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:01:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.007 162 0.006) gunicorn-registry stdout | 2022-05-05 10:01:47,386 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:01:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:01:47,386 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:01:47,387 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:01:47,388 [221] [DEBUG] [app] Starting request: urn:request:fbf40360-3a78-4f88-8ce8-ee684e7b414d (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:01:47,389 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:01:47,389 [221] [DEBUG] [app] Ending request: urn:request:fbf40360-3a78-4f88-8ce8-ee684e7b414d (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:01:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2022-05-05 10:01:47,390 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:01:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:01:47,390 [222] [DEBUG] [app] Starting request: urn:request:359071a2-40ae-4465-a71a-8e5838f5b4df (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:01:47,391 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:01:47,391 [222] [DEBUG] [app] Ending request: urn:request:359071a2-40ae-4465-a71a-8e5838f5b4df (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:01:47,391 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:01:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:01:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.001) gunicorn-web stdout | 2022-05-05 10:01:47,392 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:01:47,394 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 1, 47, 392487), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 1, 47, 392532), 'quay', None, datetime.datetime(2022, 5, 4, 10, 1, 47, 392559), datetime.datetime(2022, 4, 28, 10, 1, 47, 392579), None]) gunicorn-web stdout | 2022-05-05 10:01:47,395 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 1, 47, 393473), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 1, 47, 393517), 'quay', None, datetime.datetime(2022, 5, 4, 10, 1, 47, 393538), datetime.datetime(2022, 4, 28, 10, 1, 47, 393560), None]) gunicorn-web stdout | 2022-05-05 10:01:47,406 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:01:47,406 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:01:47,411 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:01:47,412 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:01:47,414 [221] [DEBUG] [app] Ending request: urn:request:0cdf72c0-f01a-4dd0-be78-39d1bd7a37a2 (/health/instance) gunicorn-web stdout | 2022-05-05 10:01:47,414 [222] [DEBUG] [app] Ending request: urn:request:ba5f16aa-8a2e-4800-86e7-b2ec9ff62a9b (/health/instance) gunicorn-web stdout | 2022-05-05 10:01:47,414 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:01:47,414 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:01:47,414 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:01:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:01:47,414 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:01:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:01:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.046 118 0.045) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:01:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.046 118 0.045) gunicorn-registry stdout | 2022-05-05 10:01:50,660 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:01:50,664 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:01:50,667 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:01:50,670 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:01:50,673 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:01:50,676 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:01:50,679 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:01:50,682 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:01:50,687 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:01:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:01:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:03.089985+00:00 (in 9.999504 seconds) notificationworker stdout | 2022-05-05 10:01:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:03 UTC)" (scheduled at 2022-05-05 10:01:53.089985+00:00) notificationworker stdout | 2022-05-05 10:01:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:01:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 1, 53, 91117), True, datetime.datetime(2022, 5, 5, 10, 1, 53, 91117), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:01:53,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:01:53,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:01:53,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:01:54,843 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:01:55,438 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:01:56,867 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:01:56,868 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:26.865082+00:00 (in 29.997109 seconds) gcworker stdout | 2022-05-05 10:01:56,868 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:26 UTC)" (scheduled at 2022-05-05 10:01:56.865082+00:00) gcworker stdout | 2022-05-05 10:01:56,869 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535316868, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:01:56,879 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:01:56,879 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:01:56,880 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:01:58,203 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:01:58,204 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:28.201650+00:00 (in 29.997765 seconds) securityworker stdout | 2022-05-05 10:01:58,204 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:28 UTC)" (scheduled at 2022-05-05 10:01:58.201650+00:00) securityworker stdout | 2022-05-05 10:01:58,204 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:01:58,204 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:01:58,207 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:01:58,209 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:01:58,217 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:01:58,217 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:01:59,241 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:01:59,440 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:01:59,730 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:02:00,314 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:02:00,658 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:02:00,980 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:02:00,980 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:47.187465+00:00 (in 46.207237 seconds) exportactionlogsworker stdout | 2022-05-05 10:02:00,980 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:00 UTC)" (scheduled at 2022-05-05 10:02:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:02:00,980 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:02:00,982 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 0, 980728), True, datetime.datetime(2022, 5, 5, 10, 2, 0, 980728), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:02:00,994 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:02:00,995 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:02:00,995 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:00 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:02:01,053 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} manifestbackfillworker stdout | 2022-05-05 10:02:02,339 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} gunicorn-web stdout | 2022-05-05 10:02:02,369 [221] [DEBUG] [app] Starting request: urn:request:9cec4e64-4b91-41af-ab3c-1acf964fcfb1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:02,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:02,372 [219] [DEBUG] [app] Starting request: urn:request:181c3a9c-c87a-47b9-972f-f8579666771f (/health/instance) gunicorn-registry stdout | 2022-05-05 10:02:02,374 [224] [DEBUG] [app] Starting request: urn:request:63c93882-aaf7-4b7e-be07-d774ada10abb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:02,374 [224] [DEBUG] [app] Ending request: urn:request:63c93882-aaf7-4b7e-be07-d774ada10abb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:02,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:02:02,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:02:02,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:02,378 [221] [DEBUG] [app] Starting request: urn:request:075243a7-3744-4087-ad0b-8636f384d41e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:02,379 [221] [DEBUG] [app] Ending request: urn:request:075243a7-3744-4087-ad0b-8636f384d41e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:02,379 [219] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.001) gunicorn-web stdout | 2022-05-05 10:02:02,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:02:02,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:02,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2022-05-05 10:02:02,381 [224] [DEBUG] [app] Starting request: urn:request:d8a71ae7-b565-48ff-a3c3-76f2a50b8698 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:02,381 [224] [DEBUG] [app] Ending request: urn:request:d8a71ae7-b565-48ff-a3c3-76f2a50b8698 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:02,382 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:02:02,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:02:02,384 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:02,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:02:02,387 [222] [DEBUG] [app] Starting request: urn:request:337d7b58-c273-479d-8308-db70cb60619c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:02,388 [222] [DEBUG] [app] Ending request: urn:request:337d7b58-c273-479d-8308-db70cb60619c (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:02:02,388 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:02:02,389 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:02,391 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 2, 2, 390510), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 2, 2, 390542), 'quay', None, datetime.datetime(2022, 5, 4, 10, 2, 2, 390560), datetime.datetime(2022, 4, 28, 10, 2, 2, 390575), None]) gunicorn-web stdout | 2022-05-05 10:02:02,393 [221] [DEBUG] [app] Ending request: urn:request:9cec4e64-4b91-41af-ab3c-1acf964fcfb1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:02,393 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:02:02,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:02:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:02:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:02:02,401 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:02:02,407 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:02:02,410 [219] [DEBUG] [app] Ending request: urn:request:181c3a9c-c87a-47b9-972f-f8579666771f (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:02,410 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:02:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.041 118 0.042) gunicorn-web stdout | 2022-05-05 10:02:02,411 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:02:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" builder stdout | 2022-05-05 10:02:02,461 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:02:02,802 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:02:02,805 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:02:02,808 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} buildlogsarchiver stdout | 2022-05-05 10:02:02,916 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:02:02,965 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} notificationworker stdout | 2022-05-05 10:02:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:02:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:12.207990+00:00 (in 9.117426 seconds) notificationworker stdout | 2022-05-05 10:02:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:13 UTC)" (scheduled at 2022-05-05 10:02:03.089985+00:00) notificationworker stdout | 2022-05-05 10:02:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:02:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 3, 91112), True, datetime.datetime(2022, 5, 5, 10, 2, 3, 91112), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:02:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:02:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:02:03,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:13 UTC)" executed successfully teamsyncworker stdout | 2022-05-05 10:02:03,142 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:02:03,854 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:02:04,857 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:02:05,053 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:02:05,143 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:02:05,371 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:02:05,653 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:02:05,807 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:02:05,820 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:02:05,825 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:02:05,832 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:02:05,837 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} queuecleanupworker stdout | 2022-05-05 10:02:06,554 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:02:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:02:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:13.089985+00:00 (in 0.881382 seconds) notificationworker stdout | 2022-05-05 10:02:12,209 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:12 UTC)" (scheduled at 2022-05-05 10:02:12.207990+00:00) notificationworker stdout | 2022-05-05 10:02:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:02:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:02:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:02:13,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:23.089985+00:00 (in 9.999117 seconds) notificationworker stdout | 2022-05-05 10:02:13,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:23 UTC)" (scheduled at 2022-05-05 10:02:13.089985+00:00) notificationworker stdout | 2022-05-05 10:02:13,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:02:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 13, 91337), True, datetime.datetime(2022, 5, 5, 10, 2, 13, 91337), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:02:13,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:02:13,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:02:13,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:02:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:02:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:24.984990+00:00 (in 11.589593 seconds) repositorygcworker stdout | 2022-05-05 10:02:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:13 UTC)" (scheduled at 2022-05-05 10:02:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:02:13,396 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:02:13,396 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:02:14,288 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:02:14,289 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:38.977973+00:00 (in 24.688704 seconds) namespacegcworker stdout | 2022-05-05 10:02:14,289 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:14 UTC)" (scheduled at 2022-05-05 10:02:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:02:14,289 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:02:14,290 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:02:14,482 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:02:14,483 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:44.479974+00:00 (in 29.997205 seconds) buildlogsarchiver stdout | 2022-05-05 10:02:14,483 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:44 UTC)" (scheduled at 2022-05-05 10:02:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:02:14,484 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 2, 14, 483564), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:02:14,497 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:02:14,497 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:02:14,497 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:44 UTC)" executed successfully expiredappspecifictokenworker stdout | 2022-05-05 10:02:15,481 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run expiredappspecifictokenworker stdout | 2022-05-05 10:02:15,482 [60] [INFO] [apscheduler.executors.default] Running job "ExpiredAppSpecificTokenWorker._gc_expired_tokens (trigger: interval[1:00:00], next run at: 2022-05-05 10:02:15 UTC)" (scheduled at 2022-05-05 10:02:15.480971+00:00) expiredappspecifictokenworker stdout | 2022-05-05 10:02:15,482 [60] [DEBUG] [__main__] Garbage collecting expired app specific tokens with window: 1 day, 0:00:00 expiredappspecifictokenworker stdout | 2022-05-05 10:02:15,483 [60] [DEBUG] [peewee] ('DELETE FROM "appspecificauthtoken" WHERE ("appspecificauthtoken"."expiration" < %s)', [datetime.datetime(2022, 5, 4, 10, 2, 15, 483202)]) expiredappspecifictokenworker stdout | 2022-05-05 10:02:15,484 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 11:02:15.480971+00:00 (in 3599.999586 seconds) expiredappspecifictokenworker stdout | 2022-05-05 10:02:15,491 [60] [DEBUG] [data.database] Disconnecting from database. expiredappspecifictokenworker stdout | 2022-05-05 10:02:15,491 [60] [INFO] [apscheduler.executors.default] Job "ExpiredAppSpecificTokenWorker._gc_expired_tokens (trigger: interval[1:00:00], next run at: 2022-05-05 11:02:15 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:02:17,370 [220] [DEBUG] [app] Starting request: urn:request:5f2d3752-d73d-4807-8282-3d6e7e9c4dd0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:17,370 [219] [DEBUG] [app] Starting request: urn:request:7b3e37a1-a835-477d-8762-2d78b938922a (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:17,372 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:17,373 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:02:17,377 [224] [DEBUG] [app] Starting request: urn:request:dc54f8c4-6628-46d9-b589-b1245cfa026f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:17,377 [228] [DEBUG] [app] Starting request: urn:request:7629bbe6-50a7-42ec-badb-40287b0c8177 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:17,378 [224] [DEBUG] [app] Ending request: urn:request:dc54f8c4-6628-46d9-b589-b1245cfa026f (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-registry stdout | 2022-05-05 10:02:17,378 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:02:17,379 [228] [DEBUG] [app] Ending request: urn:request:7629bbe6-50a7-42ec-badb-40287b0c8177 (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:17,378 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-registry stdout | 2022-05-05 10:02:17,380 [228] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:02:17,380 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:17,381 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:17,383 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:17,384 [219] [DEBUG] [app] Starting request: urn:request:dde6628b-bf14-419b-9ef7-5d1b52890e32 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:17,385 [219] [DEBUG] [app] Ending request: urn:request:dde6628b-bf14-419b-9ef7-5d1b52890e32 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:17,385 [220] [DEBUG] [app] Starting request: urn:request:9f62fc41-87b4-40dc-90a7-fe7ab8dc6bcc (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:17,385 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:02:17,386 [220] [DEBUG] [app] Ending request: urn:request:9f62fc41-87b4-40dc-90a7-fe7ab8dc6bcc (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:17,386 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:17,386 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:02:17,387 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:17,387 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:02:17,387 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:02:17,395 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:02:17,395 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:02:17,403 [219] [DEBUG] [app] Ending request: urn:request:7b3e37a1-a835-477d-8762-2d78b938922a (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:17,403 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:02:17,404 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:02:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:02:17,404 [220] [DEBUG] [app] Ending request: urn:request:5f2d3752-d73d-4807-8282-3d6e7e9c4dd0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:17,404 [220] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:02:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.036 118 0.035) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:02:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.035 118 0.036) gunicorn-web stdout | 2022-05-05 10:02:17,405 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:02:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:02:20,671 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:02:20,675 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:02:20,679 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:02:20,684 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:02:20,687 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:02:20,691 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:02:20,696 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:02:20,699 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:02:20,702 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:02:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:02:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:33.089985+00:00 (in 9.999422 seconds) notificationworker stdout | 2022-05-05 10:02:23,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:33 UTC)" (scheduled at 2022-05-05 10:02:23.089985+00:00) notificationworker stdout | 2022-05-05 10:02:23,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:02:23,093 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 23, 91419), True, datetime.datetime(2022, 5, 5, 10, 2, 23, 91419), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:02:23,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:02:23,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:02:23,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:33 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:02:24,882 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} repositorygcworker stdout | 2022-05-05 10:02:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:02:24,986 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:00.385766+00:00 (in 35.399971 seconds) repositorygcworker stdout | 2022-05-05 10:02:24,986 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:24 UTC)" (scheduled at 2022-05-05 10:02:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:02:24,986 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:02:24,988 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 24, 986510), True, datetime.datetime(2022, 5, 5, 10, 2, 24, 986510), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:02:24,998 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:02:24,998 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:02:24,998 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:24 UTC)" executed successfully expiredappspecifictokenworker stdout | 2022-05-05 10:02:25,479 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:02:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:02:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:56.865082+00:00 (in 29.999474 seconds) gcworker stdout | 2022-05-05 10:02:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:56 UTC)" (scheduled at 2022-05-05 10:02:26.865082+00:00) gcworker stdout | 2022-05-05 10:02:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535346866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:02:26,880 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:02:26,880 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:02:26,881 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:02:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:02:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:58.201650+00:00 (in 29.999497 seconds) securityworker stdout | 2022-05-05 10:02:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:58 UTC)" (scheduled at 2022-05-05 10:02:28.201650+00:00) securityworker stdout | 2022-05-05 10:02:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:02:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:02:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:02:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:02:28,217 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:02:28,219 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:02:28,221 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:02:28,222 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:02:28,222 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 9, 57, 28, 222145), 1, 2]) securityworker stdout | 2022-05-05 10:02:28,224 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:02:28,224 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:02:28,225 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 9, 57, 28, 224637), 1, 2]) securityworker stdout | 2022-05-05 10:02:28,227 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:02:28,227 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:02:28,227 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:02:28,227 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:02:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:02:29,278 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:02:29,470 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:02:29,738 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:02:30,339 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:02:30,665 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:02:31,089 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:02:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:02:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:35.969483+00:00 (in 4.194750 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:02:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:31 UTC)" (scheduled at 2022-05-05 10:02:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:02:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:02:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 31, 775179), True, datetime.datetime(2022, 5, 5, 10, 2, 31, 775179), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:02:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:02:31,786 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:02:31,786 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:02:32,370 [221] [DEBUG] [app] Starting request: urn:request:261b79db-85b1-4c95-b0c3-79aeebaa28a8 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:32,370 [220] [DEBUG] [app] Starting request: urn:request:8c7042bf-6151-446f-bd32-b062c2d30b22 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:32,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:32,373 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:02:32,374 [231] [DEBUG] [app] Starting request: urn:request:0fe96a24-cef9-42d0-a085-149aca1e21fb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:32,374 [231] [DEBUG] [app] Ending request: urn:request:0fe96a24-cef9-42d0-a085-149aca1e21fb (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:02:32,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:02:32,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:02:32,376 [230] [DEBUG] [app] Starting request: urn:request:eb8804d6-a0cf-4458-963f-fb8929c88166 (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:32,378 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:02:32,379 [230] [DEBUG] [app] Ending request: urn:request:eb8804d6-a0cf-4458-963f-fb8929c88166 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-registry stdout | 2022-05-05 10:02:32,380 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:02:32,380 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:32,380 [222] [DEBUG] [app] Starting request: urn:request:10ab3aae-8551-41da-9166-b20badf4797e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:32,381 [222] [DEBUG] [app] Ending request: urn:request:10ab3aae-8551-41da-9166-b20badf4797e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:32,381 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:02:32,382 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 manifestbackfillworker stdout | 2022-05-05 10:02:32,381 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} gunicorn-web stdout | 2022-05-05 10:02:32,382 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:32,383 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:02:32,384 [222] [DEBUG] [app] Starting request: urn:request:78e16fe3-9241-4149-8eae-48dd792521c6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:32,385 [222] [DEBUG] [app] Ending request: urn:request:78e16fe3-9241-4149-8eae-48dd792521c6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:32,385 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:02:32,386 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:32,387 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:02:32,391 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:02:32,394 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:02:32,402 [221] [DEBUG] [app] Ending request: urn:request:261b79db-85b1-4c95-b0c3-79aeebaa28a8 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:32,402 [220] [DEBUG] [app] Ending request: urn:request:8c7042bf-6151-446f-bd32-b062c2d30b22 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:32,402 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:02:32,402 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:02:32,403 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:02:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:02:32,403 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:02:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:02:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.034 118 0.034) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:02:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.035 118 0.034) builder stdout | 2022-05-05 10:02:32,502 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:02:32,815 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:02:32,821 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:02:32,827 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} buildlogsarchiver stdout | 2022-05-05 10:02:32,954 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:02:33,005 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} notificationworker stdout | 2022-05-05 10:02:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:02:33,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:43.089985+00:00 (in 9.999260 seconds) notificationworker stdout | 2022-05-05 10:02:33,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:43 UTC)" (scheduled at 2022-05-05 10:02:33.089985+00:00) notificationworker stdout | 2022-05-05 10:02:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:02:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 33, 91407), True, datetime.datetime(2022, 5, 5, 10, 2, 33, 91407), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:02:33,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:02:33,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:02:33,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:43 UTC)" executed successfully teamsyncworker stdout | 2022-05-05 10:02:33,171 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:02:33,894 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:02:34,897 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:02:35,092 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:02:35,184 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:02:35,411 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:02:35,662 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:02:35,817 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:02:35,829 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:02:35,835 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:02:35,839 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:02:35,845 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:02:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:02:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:03.965762+00:00 (in 27.995824 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:02:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:35 UTC)" (scheduled at 2022-05-05 10:02:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:02:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:02:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:35 UTC)" executed successfully queuecleanupworker stdout | 2022-05-05 10:02:36,570 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:02:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:02:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:14.286471+00:00 (in 35.307733 seconds) namespacegcworker stdout | 2022-05-05 10:02:38,979 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:38 UTC)" (scheduled at 2022-05-05 10:02:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:02:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:02:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 38, 979252), True, datetime.datetime(2022, 5, 5, 10, 2, 38, 979252), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:02:38,989 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:02:38,989 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:02:38,989 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:02:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:02:43,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:02:53.089985+00:00 (in 9.999133 seconds) notificationworker stdout | 2022-05-05 10:02:43,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:53 UTC)" (scheduled at 2022-05-05 10:02:43.089985+00:00) notificationworker stdout | 2022-05-05 10:02:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:02:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 43, 91354), True, datetime.datetime(2022, 5, 5, 10, 2, 43, 91354), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:02:43,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:02:43,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:02:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:02:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:02:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:02:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:14.479974+00:00 (in 29.999208 seconds) buildlogsarchiver stdout | 2022-05-05 10:02:44,481 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:14 UTC)" (scheduled at 2022-05-05 10:02:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:02:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 2, 44, 481200), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:02:44,493 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:02:44,494 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:02:44,494 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:02:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:02:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:00.978909+00:00 (in 13.791040 seconds) exportactionlogsworker stdout | 2022-05-05 10:02:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:47 UTC)" (scheduled at 2022-05-05 10:02:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:02:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:02:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:03:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:02:47,369 [222] [DEBUG] [app] Starting request: urn:request:4e735c71-39a1-4015-8479-066084a8bb48 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:47,369 [221] [DEBUG] [app] Starting request: urn:request:ca77d2cd-7ebe-42b7-b8ea-51e200b6151a (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:47,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:02:47,373 [224] [DEBUG] [app] Starting request: urn:request:9cf168bb-b06d-456b-b09a-6885d651facc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:47,373 [228] [DEBUG] [app] Starting request: urn:request:d40cbcc3-14df-44d6-b033-bcc6a394558a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:47,374 [224] [DEBUG] [app] Ending request: urn:request:9cf168bb-b06d-456b-b09a-6885d651facc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:47,374 [228] [DEBUG] [app] Ending request: urn:request:d40cbcc3-14df-44d6-b033-bcc6a394558a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:02:47,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:02:47,374 [228] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:02:47,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:47,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-web stdout | 2022-05-05 10:02:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:47,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:02:47,378 [221] [DEBUG] [app] Starting request: urn:request:3268c55c-4583-4466-a0f9-ba30645fb9f7 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:47,378 [219] [DEBUG] [app] Starting request: urn:request:43c192e5-3c45-4dc4-8a5d-d218ed0d8c0b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:47,378 [221] [DEBUG] [app] Ending request: urn:request:3268c55c-4583-4466-a0f9-ba30645fb9f7 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:47,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:02:47,379 [219] [DEBUG] [app] Ending request: urn:request:43c192e5-3c45-4dc4-8a5d-d218ed0d8c0b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:02:47,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:47,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:02:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:02:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:02:47,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:02:47,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:02:47,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:02:47,386 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:02:47,386 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:02:47,392 [222] [DEBUG] [app] Ending request: urn:request:4e735c71-39a1-4015-8479-066084a8bb48 (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:47,392 [221] [DEBUG] [app] Ending request: urn:request:ca77d2cd-7ebe-42b7-b8ea-51e200b6151a (/health/instance) gunicorn-web stdout | 2022-05-05 10:02:47,392 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:02:47,392 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:02:47,393 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:02:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:02:47,393 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:02:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:02:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.024 118 0.024) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:02:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.024 118 0.024) gunicorn-registry stdout | 2022-05-05 10:02:50,681 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:02:50,684 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:02:50,689 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:02:50,693 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:02:50,696 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:02:50,699 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:02:50,708 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:02:50,711 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:02:50,714 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:02:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:02:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:03.089985+00:00 (in 9.999492 seconds) notificationworker stdout | 2022-05-05 10:02:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:03 UTC)" (scheduled at 2022-05-05 10:02:53.089985+00:00) notificationworker stdout | 2022-05-05 10:02:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:02:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 2, 53, 90988), True, datetime.datetime(2022, 5, 5, 10, 2, 53, 90988), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:02:53,106 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:02:53,106 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:02:53,106 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:02:54,925 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:02:55,519 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:02:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:02:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:26.865082+00:00 (in 29.999355 seconds) gcworker stdout | 2022-05-05 10:02:56,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:26 UTC)" (scheduled at 2022-05-05 10:02:56.865082+00:00) gcworker stdout | 2022-05-05 10:02:56,868 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535376866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:02:56,890 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:02:56,890 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:02:56,890 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:02:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:02:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:28.201650+00:00 (in 29.999377 seconds) securityworker stdout | 2022-05-05 10:02:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:28 UTC)" (scheduled at 2022-05-05 10:02:58.201650+00:00) securityworker stdout | 2022-05-05 10:02:58,203 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:02:58,203 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:02:58,207 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:02:58,210 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:02:58,223 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:02:58,223 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:02:59,318 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:02:59,510 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:02:59,781 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:03:00,382 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} repositorygcworker stdout | 2022-05-05 10:03:00,386 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:03:00,386 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:13.394619+00:00 (in 13.008169 seconds) repositorygcworker stdout | 2022-05-05 10:03:00,387 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:08:00 UTC)" (scheduled at 2022-05-05 10:03:00.385766+00:00) repositorygcworker stdout | 2022-05-05 10:03:00,388 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 3, 0, 387321), 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:03:00,410 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 3, 0, 387321), True, datetime.datetime(2022, 5, 5, 10, 3, 0, 387321), 0, 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:03:00,413 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 3, 0, 387321), True, datetime.datetime(2022, 5, 5, 10, 3, 0, 387321), 0, 'repositorygc/%', False, datetime.datetime(2022, 5, 5, 10, 3, 0, 387321), 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:03:00,415 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:03:00,416 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:08:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:03:00,708 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:03:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:03:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:47.187465+00:00 (in 46.207838 seconds) exportactionlogsworker stdout | 2022-05-05 10:03:00,980 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:00 UTC)" (scheduled at 2022-05-05 10:03:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:03:00,980 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:03:00,982 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 0, 980471), True, datetime.datetime(2022, 5, 5, 10, 3, 0, 980471), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:03:00,993 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:03:00,993 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:03:00,993 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:00 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:03:01,128 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:03:02,370 [219] [DEBUG] [app] Starting request: urn:request:8df39cce-843a-422e-9d43-80fc7643238a (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:02,370 [221] [DEBUG] [app] Starting request: urn:request:cec298bb-08a3-4231-960a-2f11487d99cc (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:02,373 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:02,373 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:03:02,375 [230] [DEBUG] [app] Starting request: urn:request:9364a18f-2534-4a5d-9bab-6788edc7db18 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:02,376 [230] [DEBUG] [app] Ending request: urn:request:9364a18f-2534-4a5d-9bab-6788edc7db18 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.003) gunicorn-registry stdout | 2022-05-05 10:03:02,376 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:03:02,377 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:03:02,377 [225] [DEBUG] [app] Starting request: urn:request:c9e09d78-12f7-42e6-9f35-53a743838347 (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:02,379 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:03:02,380 [225] [DEBUG] [app] Ending request: urn:request:c9e09d78-12f7-42e6-9f35-53a743838347 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:02,381 [225] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:03:02,381 [220] [DEBUG] [app] Starting request: urn:request:2f70f56d-830c-4aca-9515-4b2baee38353 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:02,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:02,382 [220] [DEBUG] [app] Ending request: urn:request:2f70f56d-830c-4aca-9515-4b2baee38353 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:02,383 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:03:02,383 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.007 162 0.007) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:03:02,383 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:03:02,385 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:02,387 [220] [DEBUG] [app] Starting request: urn:request:05d1d620-1dc8-4c10-8d24-c93b301b41c0 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:02,388 [220] [DEBUG] [app] Ending request: urn:request:05d1d620-1dc8-4c10-8d24-c93b301b41c0 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:02,389 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:03:02,389 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:02,390 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:03:02,393 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:03:02,397 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:03:02,401 [221] [DEBUG] [app] Ending request: urn:request:cec298bb-08a3-4231-960a-2f11487d99cc (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:02,402 [221] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:03:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.033) gunicorn-web stdout | 2022-05-05 10:03:02,402 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:03:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:03:02,408 [219] [DEBUG] [app] Ending request: urn:request:8df39cce-843a-422e-9d43-80fc7643238a (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:02,409 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:03:02,409 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:03:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:03:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.040 118 0.040) manifestbackfillworker stdout | 2022-05-05 10:03:02,424 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:03:02,543 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:03:02,829 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:03:02,838 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:03:02,843 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} buildlogsarchiver stdout | 2022-05-05 10:03:02,998 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:03:03,046 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} notificationworker stdout | 2022-05-05 10:03:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:03:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:12.207990+00:00 (in 9.117398 seconds) notificationworker stdout | 2022-05-05 10:03:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:13 UTC)" (scheduled at 2022-05-05 10:03:03.089985+00:00) notificationworker stdout | 2022-05-05 10:03:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:03:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 3, 91131), True, datetime.datetime(2022, 5, 5, 10, 3, 3, 91131), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:03:03,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:03:03,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:03:03,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:13 UTC)" executed successfully teamsyncworker stdout | 2022-05-05 10:03:03,212 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:03:03,933 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityscanningnotificationworker stdout | 2022-05-05 10:03:03,966 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:03:03,966 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:31.774027+00:00 (in 27.807732 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:03:03,966 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:08:03 UTC)" (scheduled at 2022-05-05 10:03:03.965762+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:03:03,968 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 3, 3, 967018), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:03:03,979 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 3, 3, 967018), True, datetime.datetime(2022, 5, 5, 10, 3, 3, 967018), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:03:03,983 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 3, 3, 967018), True, datetime.datetime(2022, 5, 5, 10, 3, 3, 967018), 0, 'secscanv4/%', False, datetime.datetime(2022, 5, 5, 10, 3, 3, 967018), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:03:03,985 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:03:03,985 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:08:03 UTC)" executed successfully securityworker stdout | 2022-05-05 10:03:04,937 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:03:05,132 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:03:05,227 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:03:05,453 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:03:05,701 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:03:05,830 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:03:05,844 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:03:05,853 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:03:05,859 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:03:05,864 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} queuecleanupworker stdout | 2022-05-05 10:03:06,596 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:03:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:03:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:13.089985+00:00 (in 0.881403 seconds) notificationworker stdout | 2022-05-05 10:03:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:12 UTC)" (scheduled at 2022-05-05 10:03:12.207990+00:00) notificationworker stdout | 2022-05-05 10:03:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:03:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:03:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:03:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:23.089985+00:00 (in 9.999416 seconds) notificationworker stdout | 2022-05-05 10:03:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:23 UTC)" (scheduled at 2022-05-05 10:03:13.089985+00:00) notificationworker stdout | 2022-05-05 10:03:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:03:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 13, 91006), True, datetime.datetime(2022, 5, 5, 10, 3, 13, 91006), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:03:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:03:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:03:13,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:03:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:03:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:24.984990+00:00 (in 11.589581 seconds) repositorygcworker stdout | 2022-05-05 10:03:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:13 UTC)" (scheduled at 2022-05-05 10:03:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:03:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:03:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:03:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:03:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:38.977973+00:00 (in 24.691035 seconds) namespacegcworker stdout | 2022-05-05 10:03:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:14 UTC)" (scheduled at 2022-05-05 10:03:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:03:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:03:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:03:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:03:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:44.479974+00:00 (in 29.999515 seconds) buildlogsarchiver stdout | 2022-05-05 10:03:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:44 UTC)" (scheduled at 2022-05-05 10:03:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:03:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 3, 14, 480940), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:03:14,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:03:14,490 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:03:14,490 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:03:17,370 [219] [DEBUG] [app] Starting request: urn:request:d930d3fc-4886-4fd2-8164-5c63ff6e4c05 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:17,370 [222] [DEBUG] [app] Starting request: urn:request:dfe35678-8a94-449b-bdb2-37e165f779f4 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:17,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:17,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:03:17,373 [224] [DEBUG] [app] Starting request: urn:request:c43f362a-47d6-45f2-aec1-17118e6be508 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:17,374 [224] [DEBUG] [app] Ending request: urn:request:c43f362a-47d6-45f2-aec1-17118e6be508 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:17,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:03:17,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:03:17,376 [229] [DEBUG] [app] Starting request: urn:request:55cbb82a-90bc-43fa-b662-271273c69f0e (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:17,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:03:17,378 [229] [DEBUG] [app] Ending request: urn:request:55cbb82a-90bc-43fa-b662-271273c69f0e (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.006) gunicorn-registry stdout | 2022-05-05 10:03:17,378 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:03:17,379 [219] [DEBUG] [app] Starting request: urn:request:4c787ed8-c644-4aa6-9632-d6a220305845 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:17,379 [219] [DEBUG] [app] Ending request: urn:request:4c787ed8-c644-4aa6-9632-d6a220305845 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:17,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:03:17,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:17,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:17,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:03:17,382 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:17,383 [219] [DEBUG] [app] Starting request: urn:request:103151cb-d26d-4328-90e6-b7a15e73a7e8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:17,384 [219] [DEBUG] [app] Ending request: urn:request:103151cb-d26d-4328-90e6-b7a15e73a7e8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:17,384 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.001) gunicorn-web stdout | 2022-05-05 10:03:17,384 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:17,385 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:03:17,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:03:17,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:03:17,395 [222] [DEBUG] [app] Ending request: urn:request:dfe35678-8a94-449b-bdb2-37e165f779f4 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:17,395 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:03:17,395 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:03:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:03:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:03:17,397 [219] [DEBUG] [app] Ending request: urn:request:d930d3fc-4886-4fd2-8164-5c63ff6e4c05 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:17,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:03:17,397 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:03:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:03:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-registry stdout | 2022-05-05 10:03:20,694 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:03:20,699 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:03:20,703 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:03:20,708 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:03:20,712 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:03:20,716 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:03:20,721 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:03:20,727 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:03:20,730 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:03:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:03:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:33.089985+00:00 (in 9.999440 seconds) notificationworker stdout | 2022-05-05 10:03:23,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:33 UTC)" (scheduled at 2022-05-05 10:03:23.089985+00:00) notificationworker stdout | 2022-05-05 10:03:23,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:03:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 23, 91252), True, datetime.datetime(2022, 5, 5, 10, 3, 23, 91252), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:03:23,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:03:23,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:03:23,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:33 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:03:24,963 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} repositorygcworker stdout | 2022-05-05 10:03:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:03:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:13.394619+00:00 (in 48.409095 seconds) repositorygcworker stdout | 2022-05-05 10:03:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:24 UTC)" (scheduled at 2022-05-05 10:03:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:03:24,986 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:03:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 24, 986040), True, datetime.datetime(2022, 5, 5, 10, 3, 24, 986040), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:03:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:03:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:03:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:24 UTC)" executed successfully expiredappspecifictokenworker stdout | 2022-05-05 10:03:25,560 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:03:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:03:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:56.865082+00:00 (in 29.999374 seconds) gcworker stdout | 2022-05-05 10:03:26,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:56 UTC)" (scheduled at 2022-05-05 10:03:26.865082+00:00) gcworker stdout | 2022-05-05 10:03:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535406866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:03:26,952 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:03:26,952 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:03:26,952 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:03:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:03:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:58.201650+00:00 (in 29.999330 seconds) securityworker stdout | 2022-05-05 10:03:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:58 UTC)" (scheduled at 2022-05-05 10:03:28.201650+00:00) securityworker stdout | 2022-05-05 10:03:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:03:28,203 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:03:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:03:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:03:28,219 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:03:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:03:28,224 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:03:28,224 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:03:28,225 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 9, 58, 28, 224721), 1, 2]) securityworker stdout | 2022-05-05 10:03:28,226 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:03:28,227 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:03:28,227 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 9, 58, 28, 227176), 1, 2]) securityworker stdout | 2022-05-05 10:03:28,229 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:03:28,229 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:03:28,229 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:03:28,230 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:03:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:03:29,349 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:03:29,533 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:03:29,813 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:03:30,419 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:03:30,744 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:03:31,137 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:03:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:03:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:35.969483+00:00 (in 4.194761 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:03:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:31 UTC)" (scheduled at 2022-05-05 10:03:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:03:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:03:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 31, 775245), True, datetime.datetime(2022, 5, 5, 10, 3, 31, 775245), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:03:31,789 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:03:31,789 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:03:31,789 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:03:32,369 [221] [DEBUG] [app] Starting request: urn:request:966034be-2a0c-443f-9383-309ddf8ecd7b (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:32,369 [219] [DEBUG] [app] Starting request: urn:request:1b2a3f9c-5e26-444c-bbb4-50e014a3b625 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:32,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:32,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:03:32,373 [231] [DEBUG] [app] Starting request: urn:request:6e3f2179-8c13-4c44-9f90-9d8186175562 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:32,374 [231] [DEBUG] [app] Ending request: urn:request:6e3f2179-8c13-4c44-9f90-9d8186175562 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:32,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:03:32,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-registry stdout | 2022-05-05 10:03:32,377 [226] [DEBUG] [app] Starting request: urn:request:79a7ccc2-4250-43b9-8a9f-9854b470c8b5 (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:32,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:03:32,379 [226] [DEBUG] [app] Ending request: urn:request:79a7ccc2-4250-43b9-8a9f-9854b470c8b5 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:32,381 [226] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:03:32,379 [222] [DEBUG] [app] Starting request: urn:request:73de717a-18aa-4f87-b371-7afce50421eb (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:32,380 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:32,381 [222] [DEBUG] [app] Ending request: urn:request:73de717a-18aa-4f87-b371-7afce50421eb (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:32,381 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.008 162 0.007) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.002) gunicorn-web stdout | 2022-05-05 10:03:32,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:32,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:03:32,384 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:32,385 [222] [DEBUG] [app] Starting request: urn:request:a1b1e8a7-be57-4dab-8fb8-843a852e5db9 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:32,386 [222] [DEBUG] [app] Ending request: urn:request:a1b1e8a7-be57-4dab-8fb8-843a852e5db9 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:32,387 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:03:32,387 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:03:32,387 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:03:32,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:03:32,397 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:03:32,399 [219] [DEBUG] [app] Ending request: urn:request:1b2a3f9c-5e26-444c-bbb4-50e014a3b625 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:32,399 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:03:32,400 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:03:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:03:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.031) gunicorn-web stdout | 2022-05-05 10:03:32,405 [221] [DEBUG] [app] Ending request: urn:request:966034be-2a0c-443f-9383-309ddf8ecd7b (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:32,405 [221] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:03:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.036 118 0.037) gunicorn-web stdout | 2022-05-05 10:03:32,405 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:03:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" manifestbackfillworker stdout | 2022-05-05 10:03:32,437 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:03:32,572 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:03:32,840 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:03:32,847 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:03:32,850 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} buildlogsarchiver stdout | 2022-05-05 10:03:33,036 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:03:33,085 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} notificationworker stdout | 2022-05-05 10:03:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:03:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:43.089985+00:00 (in 9.999345 seconds) notificationworker stdout | 2022-05-05 10:03:33,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:43 UTC)" (scheduled at 2022-05-05 10:03:33.089985+00:00) notificationworker stdout | 2022-05-05 10:03:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:03:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 33, 91199), True, datetime.datetime(2022, 5, 5, 10, 3, 33, 91199), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:03:33,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:03:33,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:03:33,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:43 UTC)" executed successfully teamsyncworker stdout | 2022-05-05 10:03:33,249 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:03:33,973 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:03:34,966 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:03:35,170 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:03:35,263 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:03:35,461 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:03:35,741 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:03:35,839 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:03:35,855 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:03:35,864 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:03:35,870 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:03:35,874 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:03:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:03:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:31.774027+00:00 (in 55.803944 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:03:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:35 UTC)" (scheduled at 2022-05-05 10:03:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:03:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:03:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:35 UTC)" executed successfully queuecleanupworker stdout | 2022-05-05 10:03:36,633 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:03:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:03:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:14.286471+00:00 (in 35.308023 seconds) namespacegcworker stdout | 2022-05-05 10:03:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:38 UTC)" (scheduled at 2022-05-05 10:03:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:03:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:03:38,979 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 38, 978942), True, datetime.datetime(2022, 5, 5, 10, 3, 38, 978942), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:03:38,989 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:03:38,989 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:03:38,989 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:03:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:03:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:03:53.089985+00:00 (in 9.999414 seconds) notificationworker stdout | 2022-05-05 10:03:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:53 UTC)" (scheduled at 2022-05-05 10:03:43.089985+00:00) notificationworker stdout | 2022-05-05 10:03:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:03:43,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 43, 91078), True, datetime.datetime(2022, 5, 5, 10, 3, 43, 91078), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:03:43,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:03:43,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:03:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:03:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:03:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:03:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:14.479974+00:00 (in 29.999478 seconds) buildlogsarchiver stdout | 2022-05-05 10:03:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:14 UTC)" (scheduled at 2022-05-05 10:03:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:03:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 3, 44, 481045), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:03:44,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:03:44,492 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:03:44,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:03:47,188 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:03:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:00.978909+00:00 (in 13.790767 seconds) exportactionlogsworker stdout | 2022-05-05 10:03:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:47 UTC)" (scheduled at 2022-05-05 10:03:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:03:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:03:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:04:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:03:47,369 [219] [DEBUG] [app] Starting request: urn:request:ea6787b6-e310-4206-94a4-9da9bbbbbcd4 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:47,369 [222] [DEBUG] [app] Starting request: urn:request:912acfa5-faf7-4884-a260-e5ff2b34e620 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:47,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:47,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:03:47,374 [224] [DEBUG] [app] Starting request: urn:request:bddfe1c0-ab6d-489f-a522-71ee70576c9f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:47,374 [231] [DEBUG] [app] Starting request: urn:request:3643c320-2d04-416f-9cb9-6a58b659aff9 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:47,374 [224] [DEBUG] [app] Ending request: urn:request:bddfe1c0-ab6d-489f-a522-71ee70576c9f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:47,375 [231] [DEBUG] [app] Ending request: urn:request:3643c320-2d04-416f-9cb9-6a58b659aff9 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:03:47,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:03:47,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2022-05-05 10:03:47,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:47,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:47,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:47,379 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:03:47,381 [220] [DEBUG] [app] Starting request: urn:request:681625db-398b-4cfd-a69f-303cac961926 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:47,381 [220] [DEBUG] [app] Ending request: urn:request:681625db-398b-4cfd-a69f-303cac961926 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:47,382 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.002) gunicorn-web stdout | 2022-05-05 10:03:47,382 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:47,383 [219] [DEBUG] [app] Starting request: urn:request:2e50f361-fcd4-4905-9f53-590fa634ad48 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:03:47,383 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:03:47,384 [219] [DEBUG] [app] Ending request: urn:request:2e50f361-fcd4-4905-9f53-590fa634ad48 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:03:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:03:47,384 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:03:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:03:47,385 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:03:47,385 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:03:47,390 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:03:47,392 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:03:47,398 [222] [DEBUG] [app] Ending request: urn:request:912acfa5-faf7-4884-a260-e5ff2b34e620 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:47,398 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:03:47,399 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:03:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:03:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.031) gunicorn-web stdout | 2022-05-05 10:03:47,401 [219] [DEBUG] [app] Ending request: urn:request:ea6787b6-e310-4206-94a4-9da9bbbbbcd4 (/health/instance) gunicorn-web stdout | 2022-05-05 10:03:47,401 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:03:47,402 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:03:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:03:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.034 118 0.034) gunicorn-registry stdout | 2022-05-05 10:03:50,707 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:03:50,711 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:03:50,716 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:03:50,719 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:03:50,722 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:03:50,728 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:03:50,731 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:03:50,736 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:03:50,740 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:03:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:03:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:03.089985+00:00 (in 9.999486 seconds) notificationworker stdout | 2022-05-05 10:03:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:03 UTC)" (scheduled at 2022-05-05 10:03:53.089985+00:00) notificationworker stdout | 2022-05-05 10:03:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:03:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 3, 53, 91052), True, datetime.datetime(2022, 5, 5, 10, 3, 53, 91052), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:03:53,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:03:53,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:03:53,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:03:54,998 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:03:55,597 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:03:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:03:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:26.865082+00:00 (in 29.999491 seconds) gcworker stdout | 2022-05-05 10:03:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:26 UTC)" (scheduled at 2022-05-05 10:03:56.865082+00:00) gcworker stdout | 2022-05-05 10:03:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535436866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:03:56,881 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:03:56,881 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:03:56,881 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:03:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:03:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:28.201650+00:00 (in 29.999515 seconds) securityworker stdout | 2022-05-05 10:03:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:28 UTC)" (scheduled at 2022-05-05 10:03:58.201650+00:00) securityworker stdout | 2022-05-05 10:03:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:03:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:03:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:03:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:03:58,217 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:03:58,217 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:03:59,388 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:03:59,573 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:03:59,846 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:04:00,459 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:04:00,781 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:04:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:04:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:47.187465+00:00 (in 46.208016 seconds) exportactionlogsworker stdout | 2022-05-05 10:04:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:00 UTC)" (scheduled at 2022-05-05 10:04:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:04:00,980 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:04:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 0, 980145), True, datetime.datetime(2022, 5, 5, 10, 4, 0, 980145), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:04:00,990 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:04:00,990 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:04:00,990 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:00 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:04:01,156 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:04:02,369 [222] [DEBUG] [app] Starting request: urn:request:78008891-8e8f-4b36-8e93-bca2874c5ccf (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:02,369 [221] [DEBUG] [app] Starting request: urn:request:21461be2-6cb6-4195-b586-201a93c10cf0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:02,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:02,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:04:02,373 [224] [DEBUG] [app] Starting request: urn:request:13ca6c64-bc80-4f69-9824-b7a53cd704e2 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:02,373 [231] [DEBUG] [app] Starting request: urn:request:7e11f66e-14fe-4c28-a6d3-6d8b52cb4522 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:02,373 [224] [DEBUG] [app] Ending request: urn:request:13ca6c64-bc80-4f69-9824-b7a53cd704e2 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:04:02,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:04:02,374 [231] [DEBUG] [app] Ending request: urn:request:7e11f66e-14fe-4c28-a6d3-6d8b52cb4522 (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:02,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:04:02,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:04:02,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:04:02,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:02,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:02,378 [219] [DEBUG] [app] Starting request: urn:request:1b3e898b-9b6d-45d0-bb10-2f0e1c011bf9 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:02,378 [222] [DEBUG] [app] Starting request: urn:request:c71e2e7c-1521-4d88-a54b-07d5c8aa67d0 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:02,379 [222] [DEBUG] [app] Ending request: urn:request:c71e2e7c-1521-4d88-a54b-07d5c8aa67d0 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:02,379 [219] [DEBUG] [app] Ending request: urn:request:1b3e898b-9b6d-45d0-bb10-2f0e1c011bf9 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:02,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:04:02,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:04:02,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:02,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:02,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:04:02,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:04:02,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:04:02,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:04:02,393 [222] [DEBUG] [app] Ending request: urn:request:78008891-8e8f-4b36-8e93-bca2874c5ccf (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:02,393 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:02,394 [221] [DEBUG] [app] Ending request: urn:request:21461be2-6cb6-4195-b586-201a93c10cf0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:02,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:04:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:04:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:04:02,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:02,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:04:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:04:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.027) manifestbackfillworker stdout | 2022-05-05 10:04:02,474 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:04:02,583 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:04:02,848 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:04:02,853 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:04:02,857 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} buildlogsarchiver stdout | 2022-05-05 10:04:03,069 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} notificationworker stdout | 2022-05-05 10:04:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:04:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:12.207990+00:00 (in 9.117452 seconds) notificationworker stdout | 2022-05-05 10:04:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:13 UTC)" (scheduled at 2022-05-05 10:04:03.089985+00:00) notificationworker stdout | 2022-05-05 10:04:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:04:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 3, 91096), True, datetime.datetime(2022, 5, 5, 10, 4, 3, 91096), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:04:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:04:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:04:03,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:13 UTC)" executed successfully logrotateworker stdout | 2022-05-05 10:04:03,121 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:04:03,270 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:04:04,011 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:04:05,003 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:04:05,207 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:04:05,300 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:04:05,477 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:04:05,778 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:04:05,851 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:04:05,865 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:04:05,872 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:04:05,878 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:04:05,882 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} queuecleanupworker stdout | 2022-05-05 10:04:06,663 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:04:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:04:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:13.089985+00:00 (in 0.881419 seconds) notificationworker stdout | 2022-05-05 10:04:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:12 UTC)" (scheduled at 2022-05-05 10:04:12.207990+00:00) notificationworker stdout | 2022-05-05 10:04:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:04:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:04:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:04:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:23.089985+00:00 (in 9.999509 seconds) notificationworker stdout | 2022-05-05 10:04:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:23 UTC)" (scheduled at 2022-05-05 10:04:13.089985+00:00) notificationworker stdout | 2022-05-05 10:04:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:04:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 13, 90964), True, datetime.datetime(2022, 5, 5, 10, 4, 13, 90964), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:04:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:04:13,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:04:13,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:04:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:04:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:24.984990+00:00 (in 11.589791 seconds) repositorygcworker stdout | 2022-05-05 10:04:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:13 UTC)" (scheduled at 2022-05-05 10:04:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:04:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:04:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:04:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:04:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:23.280939+00:00 (in 8.993936 seconds) namespacegcworker stdout | 2022-05-05 10:04:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:14 UTC)" (scheduled at 2022-05-05 10:04:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:04:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:04:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:04:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:04:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:44.479974+00:00 (in 29.999411 seconds) buildlogsarchiver stdout | 2022-05-05 10:04:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:44 UTC)" (scheduled at 2022-05-05 10:04:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:04:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 4, 14, 480951), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:04:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:04:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:04:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:04:17,369 [222] [DEBUG] [app] Starting request: urn:request:c30be34c-db43-4e92-bd78-4ee101689969 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:17,369 [221] [DEBUG] [app] Starting request: urn:request:f804f89c-ab40-447e-9b9f-30e3598cfd72 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:17,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:04:17,373 [224] [DEBUG] [app] Starting request: urn:request:a034e9a9-0e28-4e24-831b-d6426a256a03 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:17,373 [229] [DEBUG] [app] Starting request: urn:request:2de70f7a-c7d9-4d24-b92e-2e00c293c125 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:17,373 [229] [DEBUG] [app] Ending request: urn:request:2de70f7a-c7d9-4d24-b92e-2e00c293c125 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:17,373 [224] [DEBUG] [app] Ending request: urn:request:a034e9a9-0e28-4e24-831b-d6426a256a03 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:17,374 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:04:17,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:04:17,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:17,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:04:17,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:17,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:17,377 [221] [DEBUG] [app] Starting request: urn:request:8e12c4ca-5199-46dd-93d0-39f3b921c837 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:17,378 [219] [DEBUG] [app] Starting request: urn:request:8dcba642-1728-4343-a457-3446ade34ebd (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:17,378 [221] [DEBUG] [app] Ending request: urn:request:8e12c4ca-5199-46dd-93d0-39f3b921c837 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:17,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:04:17,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:17,379 [219] [DEBUG] [app] Ending request: urn:request:8dcba642-1728-4343-a457-3446ade34ebd (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:17,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:04:17,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2022-05-05 10:04:17,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:17,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:04:17,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:04:17,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:04:17,393 [222] [DEBUG] [app] Ending request: urn:request:c30be34c-db43-4e92-bd78-4ee101689969 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:17,394 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:17,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:04:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:04:17,394 [221] [DEBUG] [app] Ending request: urn:request:f804f89c-ab40-447e-9b9f-30e3598cfd72 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:17,394 [221] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:04:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:04:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.027) gunicorn-web stdout | 2022-05-05 10:04:17,395 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:04:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:04:20,723 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:04:20,730 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:04:20,735 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:04:20,738 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:04:20,744 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:04:20,748 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:04:20,754 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:04:20,761 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:04:20,765 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:04:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:04:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:33.089985+00:00 (in 9.999508 seconds) notificationworker stdout | 2022-05-05 10:04:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:33 UTC)" (scheduled at 2022-05-05 10:04:23.089985+00:00) notificationworker stdout | 2022-05-05 10:04:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:04:23,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 23, 90964), True, datetime.datetime(2022, 5, 5, 10, 4, 23, 90964), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:04:23,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:04:23,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:04:23,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:33 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:04:23,281 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:04:23,281 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:38.977973+00:00 (in 15.696427 seconds) namespacegcworker stdout | 2022-05-05 10:04:23,281 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:09:23 UTC)" (scheduled at 2022-05-05 10:04:23.280939+00:00) namespacegcworker stdout | 2022-05-05 10:04:23,282 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 4, 23, 281996), 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:04:23,292 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 4, 23, 281996), True, datetime.datetime(2022, 5, 5, 10, 4, 23, 281996), 0, 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:04:23,296 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 4, 23, 281996), True, datetime.datetime(2022, 5, 5, 10, 4, 23, 281996), 0, 'namespacegc/%', False, datetime.datetime(2022, 5, 5, 10, 4, 23, 281996), 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:04:23,298 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:04:23,299 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:09:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:04:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:04:24,986 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:13.394619+00:00 (in 48.408867 seconds) repositorygcworker stdout | 2022-05-05 10:04:24,986 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:24 UTC)" (scheduled at 2022-05-05 10:04:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:04:24,986 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:04:24,988 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 24, 986495), True, datetime.datetime(2022, 5, 5, 10, 4, 24, 986495), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:04:24,999 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:04:24,999 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:04:24,999 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:04:25,015 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:04:25,636 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:04:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:04:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:56.865082+00:00 (in 29.999523 seconds) gcworker stdout | 2022-05-05 10:04:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:56 UTC)" (scheduled at 2022-05-05 10:04:26.865082+00:00) gcworker stdout | 2022-05-05 10:04:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535466866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:04:26,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:04:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:04:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:56 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:04:28,103 [219] [DEBUG] [app] Starting request: urn:request:fce0e1e6-573b-4481-93aa-00fe7578155d (/api/v1/user/notifications) gunicorn-web stdout | 2022-05-05 10:04:28,104 [219] [DEBUG] [app] User loader loading deferred user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:04:28,104 [219] [DEBUG] [auth.cookie] Loading user from cookie: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:04:28,105 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['105a42ac-0110-4aed-8579-1c532997244a', False, 1, 0]) gunicorn-web stdout | 2022-05-05 10:04:28,106 [222] [DEBUG] [app] Starting request: urn:request:72b183de-5f3f-4b5d-ad92-b5ec08c0fc0b (/api/v1/user/apptoken) gunicorn-web stdout | 2022-05-05 10:04:28,106 [222] [DEBUG] [app] User loader loading deferred user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:04:28,106 [222] [DEBUG] [auth.cookie] Loading user from cookie: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:04:28,107 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['105a42ac-0110-4aed-8579-1c532997244a', False, 1, 0]) gunicorn-web stdout | 2022-05-05 10:04:28,114 [219] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-05 10:04:28,115 [219] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-05 10:04:28,115 [219] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:04:28,115 [219] [DEBUG] [endpoints.api] Checking permission for user ibazulic gunicorn-web stdout | 2022-05-05 10:04:28,115 [219] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:04:28,115 [219] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:04:28,115 [219] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:04:28,115 [219] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:04:28,116 [222] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-05 10:04:28,116 [222] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-05 10:04:28,116 [222] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:04:28,116 [222] [DEBUG] [endpoints.api] Checking permission for user ibazulic gunicorn-web stdout | 2022-05-05 10:04:28,117 [222] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:04:28,117 [222] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:04:28,117 [222] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:04:28,117 [222] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:04:28,118 [219] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2022-05-05 10:04:28,118 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2022, 5, 8, 5, 16, 28, 117643), datetime.datetime(2022, 5, 5, 10, 4, 28, 117741)]) gunicorn-web stdout | 2022-05-05 10:04:28,121 [222] [DEBUG] [app] Ending request: urn:request:72b183de-5f3f-4b5d-ad92-b5ec08c0fc0b (/api/v1/user/apptoken) gunicorn-web stdout | 2022-05-05 10:04:28,122 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:28,122 [222] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:04:28 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:04:28 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.1" 200 38 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" (0.018 1303 0.017) gunicorn-web stdout | 2022-05-05 10:04:28,124 [219] [DEBUG] [app] Ending request: urn:request:fce0e1e6-573b-4481-93aa-00fe7578155d (/api/v1/user/notifications) gunicorn-web stdout | 2022-05-05 10:04:28,125 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:28,125 [219] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:04:28 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:04:28 +0000] "GET /api/v1/user/notifications HTTP/1.1" 200 43 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" (0.023 1294 0.024) securityworker stdout | 2022-05-05 10:04:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:04:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:58.201650+00:00 (in 29.999477 seconds) securityworker stdout | 2022-05-05 10:04:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:58 UTC)" (scheduled at 2022-05-05 10:04:28.201650+00:00) securityworker stdout | 2022-05-05 10:04:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:04:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:04:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:04:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:04:28,215 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:04:28,217 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:04:28,219 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:04:28,219 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:04:28,220 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 9, 59, 28, 219937), 1, 2]) securityworker stdout | 2022-05-05 10:04:28,221 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:04:28,221 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:04:28,222 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 9, 59, 28, 222067), 1, 2]) securityworker stdout | 2022-05-05 10:04:28,224 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:04:28,224 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:04:28,224 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:04:28,224 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:04:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:04:29,403 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:04:29,592 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:04:29,884 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:04:30,496 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:04:30,805 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:04:31,171 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:04:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:04:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:35.969483+00:00 (in 4.194888 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:04:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:31 UTC)" (scheduled at 2022-05-05 10:04:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:04:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:04:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 31, 775222), True, datetime.datetime(2022, 5, 5, 10, 4, 31, 775222), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:04:31,787 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:04:31,787 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:04:31,787 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:04:32,369 [220] [DEBUG] [app] Starting request: urn:request:d62c2607-d885-4888-9e02-82fbce32252c (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:32,369 [222] [DEBUG] [app] Starting request: urn:request:f6654271-8a81-4d5d-b9f7-2758a54043f1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:32,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:32,372 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:04:32,374 [231] [DEBUG] [app] Starting request: urn:request:ae12a1ab-2856-4a47-8621-20e6eb16060d (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:32,374 [224] [DEBUG] [app] Starting request: urn:request:2979ee4c-0e13-425b-a5b5-5989080860ca (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:32,374 [231] [DEBUG] [app] Ending request: urn:request:ae12a1ab-2856-4a47-8621-20e6eb16060d (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:32,375 [224] [DEBUG] [app] Ending request: urn:request:2979ee4c-0e13-425b-a5b5-5989080860ca (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:32,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:04:32,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:04:32,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:32,375 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-web stdout | 2022-05-05 10:04:32,378 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:32,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:32,379 [219] [DEBUG] [app] Starting request: urn:request:854a5226-d722-4f35-9c78-6da6a5bbfd99 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:32,379 [221] [DEBUG] [app] Starting request: urn:request:21d94ce0-0c5d-4020-a32c-3f93f2e5e835 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:32,380 [219] [DEBUG] [app] Ending request: urn:request:854a5226-d722-4f35-9c78-6da6a5bbfd99 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:32,380 [221] [DEBUG] [app] Ending request: urn:request:21d94ce0-0c5d-4020-a32c-3f93f2e5e835 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.001) gunicorn-web stdout | 2022-05-05 10:04:32,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:04:32,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:32,381 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:04:32,381 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:32,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:04:32,381 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:04:32,387 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:04:32,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:04:32,394 [220] [DEBUG] [app] Ending request: urn:request:d62c2607-d885-4888-9e02-82fbce32252c (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:32,394 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:32,395 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:04:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:04:32,395 [222] [DEBUG] [app] Ending request: urn:request:f6654271-8a81-4d5d-b9f7-2758a54043f1 (/health/instance) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:04:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.026) gunicorn-web stdout | 2022-05-05 10:04:32,395 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:32,396 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:04:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:04:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) manifestbackfillworker stdout | 2022-05-05 10:04:32,511 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:04:32,590 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:04:32,857 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:04:32,863 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:04:32,866 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} buildlogsarchiver stdout | 2022-05-05 10:04:33,083 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} notificationworker stdout | 2022-05-05 10:04:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:04:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:43.089985+00:00 (in 9.999534 seconds) notificationworker stdout | 2022-05-05 10:04:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:43 UTC)" (scheduled at 2022-05-05 10:04:33.089985+00:00) notificationworker stdout | 2022-05-05 10:04:33,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:04:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 33, 90963), True, datetime.datetime(2022, 5, 5, 10, 4, 33, 90963), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:04:33,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:04:33,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:04:33,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:43 UTC)" executed successfully logrotateworker stdout | 2022-05-05 10:04:33,158 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:04:33,301 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:04:34,051 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:04:35,044 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:04:35,248 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:04:35,340 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:04:35,518 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:04:35,819 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:04:35,863 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:04:35,874 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:04:35,880 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:04:35,890 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:04:35,895 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:04:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:04:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:31.774027+00:00 (in 55.803984 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:04:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:35 UTC)" (scheduled at 2022-05-05 10:04:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:04:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:04:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:35 UTC)" executed successfully queuecleanupworker stdout | 2022-05-05 10:04:36,680 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:04:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:04:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:14.286471+00:00 (in 35.308009 seconds) namespacegcworker stdout | 2022-05-05 10:04:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:38 UTC)" (scheduled at 2022-05-05 10:04:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:04:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:04:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 38, 978962), True, datetime.datetime(2022, 5, 5, 10, 4, 38, 978962), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:04:38,989 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:04:38,989 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:04:38,989 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:04:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:04:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:04:53.089985+00:00 (in 9.999490 seconds) notificationworker stdout | 2022-05-05 10:04:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:53 UTC)" (scheduled at 2022-05-05 10:04:43.089985+00:00) notificationworker stdout | 2022-05-05 10:04:43,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:04:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 43, 91006), True, datetime.datetime(2022, 5, 5, 10, 4, 43, 91006), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:04:43,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:04:43,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:04:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:04:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:04:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:04:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:14.479974+00:00 (in 29.999454 seconds) buildlogsarchiver stdout | 2022-05-05 10:04:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:14 UTC)" (scheduled at 2022-05-05 10:04:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:04:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 4, 44, 481078), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:04:44,495 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:04:44,495 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:04:44,495 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:04:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:04:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:00.978909+00:00 (in 13.790776 seconds) exportactionlogsworker stdout | 2022-05-05 10:04:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:47 UTC)" (scheduled at 2022-05-05 10:04:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:04:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:04:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:05:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:04:47,369 [221] [DEBUG] [app] Starting request: urn:request:97aa60ba-6f63-42fd-8f32-f648e104f853 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:47,369 [222] [DEBUG] [app] Starting request: urn:request:4f85125c-9e44-4b6a-a732-03ba03d548ad (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:47,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:04:47,373 [224] [DEBUG] [app] Starting request: urn:request:8d716be3-5329-4a4f-97cb-af36683264af (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:47,373 [231] [DEBUG] [app] Starting request: urn:request:56c4608e-fe49-4686-8188-0b9deb07a623 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:47,373 [224] [DEBUG] [app] Ending request: urn:request:8d716be3-5329-4a4f-97cb-af36683264af (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:47,373 [231] [DEBUG] [app] Ending request: urn:request:56c4608e-fe49-4686-8188-0b9deb07a623 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:04:47,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:04:47,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:04:47,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:47,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:47,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:04:47,377 [222] [DEBUG] [app] Starting request: urn:request:d5f57c09-86f6-4b18-a26e-e5d216b3f911 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:47,377 [219] [DEBUG] [app] Starting request: urn:request:e5ae3bd5-6602-45cd-907f-1f923ba5ed64 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:47,378 [222] [DEBUG] [app] Ending request: urn:request:d5f57c09-86f6-4b18-a26e-e5d216b3f911 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:47,378 [219] [DEBUG] [app] Ending request: urn:request:e5ae3bd5-6602-45cd-907f-1f923ba5ed64 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:04:47,378 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:04:47,378 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:47,378 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:04:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:04:47,378 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:04:47,379 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:04:47,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:04:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:04:47,385 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:04:47,385 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:04:47,395 [221] [DEBUG] [app] Ending request: urn:request:97aa60ba-6f63-42fd-8f32-f648e104f853 (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:47,395 [222] [DEBUG] [app] Ending request: urn:request:4f85125c-9e44-4b6a-a732-03ba03d548ad (/health/instance) gunicorn-web stdout | 2022-05-05 10:04:47,395 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:47,395 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:04:47,396 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:04:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:04:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.028) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:04:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-web stdout | 2022-05-05 10:04:47,396 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:04:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:04:50,740 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:04:50,745 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:04:50,749 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:04:50,752 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:04:50,755 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:04:50,759 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:04:50,763 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:04:50,768 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:04:50,774 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:04:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:04:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:03.089985+00:00 (in 9.999497 seconds) notificationworker stdout | 2022-05-05 10:04:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:03 UTC)" (scheduled at 2022-05-05 10:04:53.089985+00:00) notificationworker stdout | 2022-05-05 10:04:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:04:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 4, 53, 91042), True, datetime.datetime(2022, 5, 5, 10, 4, 53, 91042), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:04:53,105 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:04:53,105 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:04:53,105 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:04:55,038 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:04:55,673 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:04:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:04:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:26.865082+00:00 (in 29.999523 seconds) gcworker stdout | 2022-05-05 10:04:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:26 UTC)" (scheduled at 2022-05-05 10:04:56.865082+00:00) gcworker stdout | 2022-05-05 10:04:56,866 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535496866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:04:56,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:04:56,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:04:56,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:04:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:04:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:28.201650+00:00 (in 29.999520 seconds) securityworker stdout | 2022-05-05 10:04:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:28 UTC)" (scheduled at 2022-05-05 10:04:58.201650+00:00) securityworker stdout | 2022-05-05 10:04:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:04:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:04:58,205 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:04:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:04:58,215 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:04:58,215 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:04:59,440 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:04:59,630 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:04:59,925 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:05:00,537 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:05:00,836 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:05:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:05:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:47.187465+00:00 (in 46.207981 seconds) exportactionlogsworker stdout | 2022-05-05 10:05:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:00 UTC)" (scheduled at 2022-05-05 10:05:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:05:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:05:00,980 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 0, 979974), True, datetime.datetime(2022, 5, 5, 10, 5, 0, 979974), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:05:00,993 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:05:00,993 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:05:00,993 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:00 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:05:01,213 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:05:02,369 [219] [DEBUG] [app] Starting request: urn:request:589889a2-b1fc-4b05-a9b5-0797adbb21c9 (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:02,369 [222] [DEBUG] [app] Starting request: urn:request:a411577a-d8a1-449a-b5c1-dccc5083ac7d (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:02,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:02,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:05:02,373 [224] [DEBUG] [app] Starting request: urn:request:89028f33-82c0-4012-9e27-7bbcc6aa0faf (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:02,373 [231] [DEBUG] [app] Starting request: urn:request:0a9fff3a-10b5-4661-ab3b-1eac0fb08b0e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:02,374 [224] [DEBUG] [app] Ending request: urn:request:89028f33-82c0-4012-9e27-7bbcc6aa0faf (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:02,374 [231] [DEBUG] [app] Ending request: urn:request:0a9fff3a-10b5-4661-ab3b-1eac0fb08b0e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:02,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:05:02,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:05:02,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:02,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:02,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:02,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:02,378 [222] [DEBUG] [app] Starting request: urn:request:55ed650f-1633-4fea-b0d4-4ab9aec6f0ee (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:02,379 [222] [DEBUG] [app] Ending request: urn:request:55ed650f-1633-4fea-b0d4-4ab9aec6f0ee (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:02,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:05:02,379 [221] [DEBUG] [app] Starting request: urn:request:bf16b1cf-8363-4d83-83b0-b57dde163894 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:02,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:02,380 [221] [DEBUG] [app] Ending request: urn:request:bf16b1cf-8363-4d83-83b0-b57dde163894 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:02,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:05:02,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:05:02,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:02,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:05:02,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:05:02,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:05:02,398 [222] [DEBUG] [app] Ending request: urn:request:a411577a-d8a1-449a-b5c1-dccc5083ac7d (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:02,398 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:05:02,399 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:05:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:05:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.030) gunicorn-web stdout | 2022-05-05 10:05:02,400 [219] [DEBUG] [app] Ending request: urn:request:589889a2-b1fc-4b05-a9b5-0797adbb21c9 (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:02,400 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:05:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.032 118 0.032) gunicorn-web stdout | 2022-05-05 10:05:02,401 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:05:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" manifestbackfillworker stdout | 2022-05-05 10:05:02,551 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:05:02,629 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:05:02,869 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:05:02,874 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:05:02,879 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:05:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:05:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:12.207990+00:00 (in 9.117483 seconds) notificationworker stdout | 2022-05-05 10:05:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:13 UTC)" (scheduled at 2022-05-05 10:05:03.089985+00:00) notificationworker stdout | 2022-05-05 10:05:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:05:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 3, 91074), True, datetime.datetime(2022, 5, 5, 10, 5, 3, 91074), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:05:03,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:05:03,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:05:03,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:13 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:05:03,120 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:05:03,199 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:05:03,331 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:05:04,087 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:05:05,051 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:05:05,255 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:05:05,365 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:05:05,539 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:05:05,854 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:05:05,872 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:05:05,883 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:05:05,888 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:05:05,898 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:05:05,901 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} queuecleanupworker stdout | 2022-05-05 10:05:06,711 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:05:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:05:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:13.089985+00:00 (in 0.881444 seconds) notificationworker stdout | 2022-05-05 10:05:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:12 UTC)" (scheduled at 2022-05-05 10:05:12.207990+00:00) notificationworker stdout | 2022-05-05 10:05:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:05:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:05:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:05:13,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:23.089985+00:00 (in 9.999338 seconds) notificationworker stdout | 2022-05-05 10:05:13,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:23 UTC)" (scheduled at 2022-05-05 10:05:13.089985+00:00) notificationworker stdout | 2022-05-05 10:05:13,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:05:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 13, 91415), True, datetime.datetime(2022, 5, 5, 10, 5, 13, 91415), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:05:13,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:05:13,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:05:13,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:05:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:05:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:24.984990+00:00 (in 11.589548 seconds) repositorygcworker stdout | 2022-05-05 10:05:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:13 UTC)" (scheduled at 2022-05-05 10:05:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:05:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:05:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:05:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:05:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:38.977973+00:00 (in 24.690994 seconds) namespacegcworker stdout | 2022-05-05 10:05:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:14 UTC)" (scheduled at 2022-05-05 10:05:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:05:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:05:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:05:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:05:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:44.479974+00:00 (in 29.999411 seconds) buildlogsarchiver stdout | 2022-05-05 10:05:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:44 UTC)" (scheduled at 2022-05-05 10:05:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:05:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 5, 14, 481036), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:05:14,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:05:14,490 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:05:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:05:17,370 [222] [DEBUG] [app] Starting request: urn:request:74db729b-ded1-4710-b72f-ffe326b28be9 (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:17,370 [221] [DEBUG] [app] Starting request: urn:request:99cc12be-a36c-4f45-a7ce-7658c929ff4f (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:17,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:05:17,373 [226] [DEBUG] [app] Starting request: urn:request:9fab373e-4c09-49c1-b8ed-ef3a79ce157b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:17,373 [224] [DEBUG] [app] Starting request: urn:request:862a96bf-243d-4a75-8994-c71a0537b01c (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:17,374 [226] [DEBUG] [app] Ending request: urn:request:9fab373e-4c09-49c1-b8ed-ef3a79ce157b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:17,374 [224] [DEBUG] [app] Ending request: urn:request:862a96bf-243d-4a75-8994-c71a0537b01c (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:17,374 [226] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:05:17,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:05:17,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:05:17,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:17,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:17,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:17,378 [220] [DEBUG] [app] Starting request: urn:request:9ab15ca1-a79a-4e4f-a4b7-7c2bc0c7eb61 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:17,379 [220] [DEBUG] [app] Ending request: urn:request:9ab15ca1-a79a-4e4f-a4b7-7c2bc0c7eb61 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:17,379 [219] [DEBUG] [app] Starting request: urn:request:3c5bbbb5-9d8c-4de6-9111-18ed098a7294 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:17,379 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:05:17,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:17,380 [219] [DEBUG] [app] Ending request: urn:request:3c5bbbb5-9d8c-4de6-9111-18ed098a7294 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:17,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:05:17,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:05:17,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:05:17,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:05:17,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:05:17,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:05:17,395 [221] [DEBUG] [app] Ending request: urn:request:99cc12be-a36c-4f45-a7ce-7658c929ff4f (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:17,395 [221] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:05:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.027) gunicorn-web stdout | 2022-05-05 10:05:17,395 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:05:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:05:17,395 [222] [DEBUG] [app] Ending request: urn:request:74db729b-ded1-4710-b72f-ffe326b28be9 (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:17,396 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:05:17,396 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:05:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:05:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-registry stdout | 2022-05-05 10:05:20,754 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:05:20,759 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:05:20,763 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:05:20,767 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:05:20,771 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:05:20,780 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:05:20,783 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:05:20,788 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:05:20,793 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:05:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:05:23,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:32.207092+00:00 (in 9.116180 seconds) notificationworker stdout | 2022-05-05 10:05:23,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:33 UTC)" (scheduled at 2022-05-05 10:05:23.089985+00:00) notificationworker stdout | 2022-05-05 10:05:23,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:05:23,094 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 23, 91450), True, datetime.datetime(2022, 5, 5, 10, 5, 23, 91450), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:05:23,105 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:05:23,105 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:05:23,105 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:05:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:05:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:13.394619+00:00 (in 48.409062 seconds) repositorygcworker stdout | 2022-05-05 10:05:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:24 UTC)" (scheduled at 2022-05-05 10:05:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:05:24,986 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:05:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 24, 986112), True, datetime.datetime(2022, 5, 5, 10, 5, 24, 986112), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:05:24,999 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:05:24,999 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:05:24,999 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:05:25,080 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:05:25,711 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:05:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:05:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:56.865082+00:00 (in 29.999513 seconds) gcworker stdout | 2022-05-05 10:05:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:56 UTC)" (scheduled at 2022-05-05 10:05:26.865082+00:00) gcworker stdout | 2022-05-05 10:05:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535526866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:05:26,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:05:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:05:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:05:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:05:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:58.201650+00:00 (in 29.999505 seconds) securityworker stdout | 2022-05-05 10:05:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:58 UTC)" (scheduled at 2022-05-05 10:05:28.201650+00:00) securityworker stdout | 2022-05-05 10:05:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:05:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:05:28,205 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:05:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:05:28,216 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:05:28,218 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:05:28,220 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:05:28,220 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:05:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 0, 28, 221084), 1, 2]) securityworker stdout | 2022-05-05 10:05:28,228 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:05:28,228 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:05:28,229 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 0, 28, 228471), 1, 2]) securityworker stdout | 2022-05-05 10:05:28,231 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:05:28,231 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:05:28,231 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:05:28,231 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:05:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:05:29,473 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:05:29,667 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:05:29,961 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:05:30,565 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:05:30,873 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:05:31,242 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:05:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:05:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:35.969483+00:00 (in 4.194987 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:05:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:31 UTC)" (scheduled at 2022-05-05 10:05:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:05:31,774 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:05:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 31, 774988), True, datetime.datetime(2022, 5, 5, 10, 5, 31, 774988), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:05:31,785 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:05:31,785 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:05:31,785 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:31 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:05:32,207 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:05:32,207 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:33.089985+00:00 (in 0.882309 seconds) notificationworker stdout | 2022-05-05 10:05:32,207 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:10:32 UTC)" (scheduled at 2022-05-05 10:05:32.207092+00:00) notificationworker stdout | 2022-05-05 10:05:32,208 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 5, 32, 208045), 'notification/%']) notificationworker stdout | 2022-05-05 10:05:32,216 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 5, 32, 208045), True, datetime.datetime(2022, 5, 5, 10, 5, 32, 208045), 0, 'notification/%']) notificationworker stdout | 2022-05-05 10:05:32,219 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 5, 32, 208045), True, datetime.datetime(2022, 5, 5, 10, 5, 32, 208045), 0, 'notification/%', False, datetime.datetime(2022, 5, 5, 10, 5, 32, 208045), 'notification/%']) notificationworker stdout | 2022-05-05 10:05:32,221 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:05:32,221 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:10:32 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:05:32,370 [222] [DEBUG] [app] Starting request: urn:request:35b9cb1e-62a5-4dd5-b83a-3988fbc6a106 (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:32,370 [219] [DEBUG] [app] Starting request: urn:request:9939a39f-4bae-45fe-80d3-4722f1529a3d (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:32,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:32,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:05:32,374 [224] [DEBUG] [app] Starting request: urn:request:1cb10acb-6e34-4308-ab15-fa1635805243 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:32,374 [229] [DEBUG] [app] Starting request: urn:request:c72bf283-a9b6-4e1f-97cb-1089f7f5c2ed (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:32,375 [229] [DEBUG] [app] Ending request: urn:request:c72bf283-a9b6-4e1f-97cb-1089f7f5c2ed (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:32,375 [224] [DEBUG] [app] Ending request: urn:request:1cb10acb-6e34-4308-ab15-fa1635805243 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:32,375 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:05:32,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-web stdout | 2022-05-05 10:05:32,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:32,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:32,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:32,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:32,378 [222] [DEBUG] [app] Starting request: urn:request:5e74b38e-a451-46ee-bd8f-ba6223e9c34a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:32,379 [222] [DEBUG] [app] Ending request: urn:request:5e74b38e-a451-46ee-bd8f-ba6223e9c34a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:32,379 [221] [DEBUG] [app] Starting request: urn:request:0ff0973c-e440-448c-beda-816701b9d938 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:32,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:05:32,380 [221] [DEBUG] [app] Ending request: urn:request:0ff0973c-e440-448c-beda-816701b9d938 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:32,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:32,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:05:32,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:32,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:05:32,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:05:32,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:05:32,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:05:32,395 [219] [DEBUG] [app] Ending request: urn:request:9939a39f-4bae-45fe-80d3-4722f1529a3d (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:32,395 [222] [DEBUG] [app] Ending request: urn:request:35b9cb1e-62a5-4dd5-b83a-3988fbc6a106 (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:32,395 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:05:32,396 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:05:32,396 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:05:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:05:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:05:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-web stdout | 2022-05-05 10:05:32,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:05:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" manifestbackfillworker stdout | 2022-05-05 10:05:32,577 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:05:32,666 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:05:32,876 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:05:32,884 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:05:32,890 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:05:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:05:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:43.089985+00:00 (in 9.999467 seconds) notificationworker stdout | 2022-05-05 10:05:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:43 UTC)" (scheduled at 2022-05-05 10:05:33.089985+00:00) notificationworker stdout | 2022-05-05 10:05:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:05:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 33, 91124), True, datetime.datetime(2022, 5, 5, 10, 5, 33, 91124), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:05:33,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:05:33,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:05:33,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:43 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:05:33,160 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:05:33,236 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:05:33,368 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:05:34,126 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:05:35,087 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:05:35,294 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:05:35,392 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:05:35,565 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:05:35,883 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} securityscanningnotificationworker stdout | 2022-05-05 10:05:35,887 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} gunicorn-web stdout | 2022-05-05 10:05:35,893 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:05:35,897 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:05:35,910 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:05:35,913 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:05:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:05:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:31.774027+00:00 (in 55.804046 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:05:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:35 UTC)" (scheduled at 2022-05-05 10:05:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:05:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:05:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:35 UTC)" executed successfully queuecleanupworker stdout | 2022-05-05 10:05:36,740 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:05:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:05:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:14.286471+00:00 (in 35.308017 seconds) namespacegcworker stdout | 2022-05-05 10:05:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:38 UTC)" (scheduled at 2022-05-05 10:05:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:05:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:05:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 38, 978907), True, datetime.datetime(2022, 5, 5, 10, 5, 38, 978907), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:05:38,991 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:05:38,991 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:05:38,991 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:05:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:05:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:53.089985+00:00 (in 9.999368 seconds) notificationworker stdout | 2022-05-05 10:05:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:53 UTC)" (scheduled at 2022-05-05 10:05:43.089985+00:00) notificationworker stdout | 2022-05-05 10:05:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:05:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 43, 91117), True, datetime.datetime(2022, 5, 5, 10, 5, 43, 91117), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:05:43,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:05:43,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:05:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:05:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:05:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:05:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:14.479974+00:00 (in 29.999384 seconds) buildlogsarchiver stdout | 2022-05-05 10:05:44,481 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:14 UTC)" (scheduled at 2022-05-05 10:05:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:05:44,482 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 5, 44, 481288), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:05:44,498 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:05:44,498 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:05:44,498 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:05:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:05:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:05:59.184038+00:00 (in 11.996018 seconds) exportactionlogsworker stdout | 2022-05-05 10:05:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:47 UTC)" (scheduled at 2022-05-05 10:05:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:05:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:05:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:06:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:05:47,370 [222] [DEBUG] [app] Starting request: urn:request:e84fefa2-da6d-4ee7-81b9-c4e619a92e00 (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:47,370 [221] [DEBUG] [app] Starting request: urn:request:583bb806-2d70-4b2f-83a9-669ed425439d (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:47,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:47,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:05:47,375 [224] [DEBUG] [app] Starting request: urn:request:f9b903c3-2f45-4a9a-8c81-2b39e04a653b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:47,375 [226] [DEBUG] [app] Starting request: urn:request:4e198fd0-e616-4d4e-955b-3ef42024a191 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:47,376 [224] [DEBUG] [app] Ending request: urn:request:f9b903c3-2f45-4a9a-8c81-2b39e04a653b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:47,376 [226] [DEBUG] [app] Ending request: urn:request:4e198fd0-e616-4d4e-955b-3ef42024a191 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:05:47,376 [226] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:05:47,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:47,377 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:47,379 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:05:47,380 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:05:47,380 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:05:47,382 [222] [DEBUG] [app] Starting request: urn:request:2dc5b5bc-7c04-49ee-8b33-2ec9e9e7bdb8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:47,382 [219] [DEBUG] [app] Starting request: urn:request:cf220f9a-84dd-4478-9bfe-e90fdc6bd66f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:47,383 [222] [DEBUG] [app] Ending request: urn:request:2dc5b5bc-7c04-49ee-8b33-2ec9e9e7bdb8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:47,383 [219] [DEBUG] [app] Ending request: urn:request:cf220f9a-84dd-4478-9bfe-e90fdc6bd66f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:05:47,383 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:05:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:05:47,383 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:05:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:05:47,384 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:47,384 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:05:47,384 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:05:47,385 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:05:47,392 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:05:47,393 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:05:47,403 [222] [DEBUG] [app] Ending request: urn:request:e84fefa2-da6d-4ee7-81b9-c4e619a92e00 (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:47,403 [221] [DEBUG] [app] Ending request: urn:request:583bb806-2d70-4b2f-83a9-669ed425439d (/health/instance) gunicorn-web stdout | 2022-05-05 10:05:47,403 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:05:47,403 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:05:47,403 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:05:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:05:47,403 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:05:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:05:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.034 118 0.034) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:05:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.034 118 0.035) gunicorn-registry stdout | 2022-05-05 10:05:50,765 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:05:50,773 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:05:50,782 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:05:50,786 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:05:50,790 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:05:50,793 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:05:50,798 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:05:50,805 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:05:50,811 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:05:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:05:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:03.089985+00:00 (in 9.999485 seconds) notificationworker stdout | 2022-05-05 10:05:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:03 UTC)" (scheduled at 2022-05-05 10:05:53.089985+00:00) notificationworker stdout | 2022-05-05 10:05:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:05:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 5, 53, 90987), True, datetime.datetime(2022, 5, 5, 10, 5, 53, 90987), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:05:53,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:05:53,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:05:53,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:05:55,107 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:05:55,735 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:05:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:05:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:26.865082+00:00 (in 29.999517 seconds) gcworker stdout | 2022-05-05 10:05:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:26 UTC)" (scheduled at 2022-05-05 10:05:56.865082+00:00) gcworker stdout | 2022-05-05 10:05:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535556866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:05:56,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:05:56,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:05:56,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:05:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:05:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:28.201650+00:00 (in 29.999517 seconds) securityworker stdout | 2022-05-05 10:05:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:28 UTC)" (scheduled at 2022-05-05 10:05:58.201650+00:00) securityworker stdout | 2022-05-05 10:05:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:05:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:05:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:05:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:05:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:05:58,216 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:28 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:05:59,184 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:05:59,184 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:00.978909+00:00 (in 1.794142 seconds) exportactionlogsworker stdout | 2022-05-05 10:05:59,185 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:10:59 UTC)" (scheduled at 2022-05-05 10:05:59.184038+00:00) exportactionlogsworker stdout | 2022-05-05 10:05:59,186 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 5, 59, 185416), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:05:59,195 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 5, 59, 185416), True, datetime.datetime(2022, 5, 5, 10, 5, 59, 185416), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:05:59,198 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 5, 59, 185416), True, datetime.datetime(2022, 5, 5, 10, 5, 59, 185416), 0, 'exportactionlogs/%', False, datetime.datetime(2022, 5, 5, 10, 5, 59, 185416), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:05:59,200 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:05:59,200 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:10:59 UTC)" executed successfully gcworker stdout | 2022-05-05 10:05:59,509 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:05:59,704 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:05:59,997 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:06:00,592 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:06:00,897 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:06:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:06:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:47.187465+00:00 (in 46.208011 seconds) exportactionlogsworker stdout | 2022-05-05 10:06:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:00 UTC)" (scheduled at 2022-05-05 10:06:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:06:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:06:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 0, 979947), True, datetime.datetime(2022, 5, 5, 10, 6, 0, 979947), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:06:00,991 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:06:00,991 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:06:00,991 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:00 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:06:01,258 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:06:02,369 [219] [DEBUG] [app] Starting request: urn:request:c25b06b6-17b2-4f84-ab5e-ad3fde599a92 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:02,369 [222] [DEBUG] [app] Starting request: urn:request:ee05e1cd-0809-4ffc-86b8-5a01e58e837e (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:02,370 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:02,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:06:02,372 [231] [DEBUG] [app] Starting request: urn:request:a06090fc-7214-44dc-9313-672de09a36da (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:02,373 [231] [DEBUG] [app] Ending request: urn:request:a06090fc-7214-44dc-9313-672de09a36da (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:02,373 [230] [DEBUG] [app] Starting request: urn:request:eb89c12d-389d-4386-a1f5-11b8b4b026fc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:02,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:06:02,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:06:02,374 [230] [DEBUG] [app] Ending request: urn:request:eb89c12d-389d-4386-a1f5-11b8b4b026fc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:02,375 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:02,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:06:02,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:02,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:02,378 [221] [DEBUG] [app] Starting request: urn:request:73723657-6fdb-41d8-9eff-4c58a32bcf93 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:02,378 [221] [DEBUG] [app] Ending request: urn:request:73723657-6fdb-41d8-9eff-4c58a32bcf93 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:06:02,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:02,379 [219] [DEBUG] [app] Starting request: urn:request:c15cdba6-710a-42b6-b153-c613210366d1 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:02,379 [219] [DEBUG] [app] Ending request: urn:request:c15cdba6-710a-42b6-b153-c613210366d1 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:02,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:06:02,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:02,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:02,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:06:02,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:06:02,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:06:02,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:06:02,394 [222] [DEBUG] [app] Ending request: urn:request:ee05e1cd-0809-4ffc-86b8-5a01e58e837e (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:02,394 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:06:02,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:06:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:06:02,394 [219] [DEBUG] [app] Ending request: urn:request:c25b06b6-17b2-4f84-ab5e-ad3fde599a92 (/health/instance) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:06:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:06:02,395 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:06:02,395 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:06:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:06:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) manifestbackfillworker stdout | 2022-05-05 10:06:02,614 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:06:02,703 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:06:02,886 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:06:02,891 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:06:02,897 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:06:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:06:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:12.207990+00:00 (in 9.117496 seconds) notificationworker stdout | 2022-05-05 10:06:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:13 UTC)" (scheduled at 2022-05-05 10:06:03.089985+00:00) notificationworker stdout | 2022-05-05 10:06:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:06:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 3, 91136), True, datetime.datetime(2022, 5, 5, 10, 6, 3, 91136), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:06:03,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:06:03,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:06:03,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:13 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:06:03,193 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:06:03,274 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:06:03,404 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:06:04,154 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:06:05,127 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:06:05,331 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:06:05,431 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:06:05,602 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:06:05,894 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:06:05,905 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:06:05,909 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:06:05,924 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:06:05,929 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:06:05,932 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:06:06,780 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:06:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:06:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:13.089985+00:00 (in 0.881413 seconds) notificationworker stdout | 2022-05-05 10:06:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:12 UTC)" (scheduled at 2022-05-05 10:06:12.207990+00:00) notificationworker stdout | 2022-05-05 10:06:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:06:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:06:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:06:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:23.089985+00:00 (in 9.999498 seconds) notificationworker stdout | 2022-05-05 10:06:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:23 UTC)" (scheduled at 2022-05-05 10:06:13.089985+00:00) notificationworker stdout | 2022-05-05 10:06:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:06:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 13, 90962), True, datetime.datetime(2022, 5, 5, 10, 6, 13, 90962), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:06:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:06:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:06:13,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:06:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:06:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:24.984990+00:00 (in 11.589775 seconds) repositorygcworker stdout | 2022-05-05 10:06:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:13 UTC)" (scheduled at 2022-05-05 10:06:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:06:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:06:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:06:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:06:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:38.977973+00:00 (in 24.690981 seconds) namespacegcworker stdout | 2022-05-05 10:06:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:14 UTC)" (scheduled at 2022-05-05 10:06:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:06:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:06:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:06:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:06:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:44.479974+00:00 (in 29.999373 seconds) buildlogsarchiver stdout | 2022-05-05 10:06:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:44 UTC)" (scheduled at 2022-05-05 10:06:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:06:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 6, 14, 481082), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:06:14,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:06:14,492 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:06:14,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:06:17,369 [219] [DEBUG] [app] Starting request: urn:request:b3b04f8d-c02b-451d-838a-8d61b342a3b0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:17,369 [222] [DEBUG] [app] Starting request: urn:request:241248ed-0d84-4ba4-8d8e-e464a12d64c8 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:17,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:17,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:06:17,374 [226] [DEBUG] [app] Starting request: urn:request:40142f8a-6f05-4f28-bd60-72f0003ab4df (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:17,374 [230] [DEBUG] [app] Starting request: urn:request:1d44b643-0683-4b19-b73f-0ddf531ec54f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:17,374 [226] [DEBUG] [app] Ending request: urn:request:40142f8a-6f05-4f28-bd60-72f0003ab4df (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:17,374 [230] [DEBUG] [app] Ending request: urn:request:1d44b643-0683-4b19-b73f-0ddf531ec54f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:17,375 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:17,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-web stdout | 2022-05-05 10:06:17,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-registry stdout | 2022-05-05 10:06:17,379 [226] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:17,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:17,380 [221] [DEBUG] [app] Starting request: urn:request:e9e38a8d-4d4d-46a3-8b08-7f29d95ff4c5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:17,380 [221] [DEBUG] [app] Ending request: urn:request:e9e38a8d-4d4d-46a3-8b08-7f29d95ff4c5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:17,381 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:17,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:17,382 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:17,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2022-05-05 10:06:17,384 [220] [DEBUG] [app] Starting request: urn:request:a9d8cf65-140b-45d9-ab04-807218719380 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:17,384 [220] [DEBUG] [app] Ending request: urn:request:a9d8cf65-140b-45d9-ab04-807218719380 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:06:17,385 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:17,385 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:17,386 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:06:17,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:06:17,392 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:06:17,397 [219] [DEBUG] [app] Ending request: urn:request:b3b04f8d-c02b-451d-838a-8d61b342a3b0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:17,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:06:17,397 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:06:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:06:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.029) gunicorn-web stdout | 2022-05-05 10:06:17,401 [222] [DEBUG] [app] Ending request: urn:request:241248ed-0d84-4ba4-8d8e-e464a12d64c8 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:17,401 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:06:17,402 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:06:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:06:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.033) gunicorn-registry stdout | 2022-05-05 10:06:20,777 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:06:20,787 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:06:20,799 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:06:20,803 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:06:20,808 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:06:20,812 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:06:20,817 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:06:20,822 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:06:20,827 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:06:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:06:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:33.089985+00:00 (in 9.999520 seconds) notificationworker stdout | 2022-05-05 10:06:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:33 UTC)" (scheduled at 2022-05-05 10:06:23.089985+00:00) notificationworker stdout | 2022-05-05 10:06:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:06:23,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 23, 90969), True, datetime.datetime(2022, 5, 5, 10, 6, 23, 90969), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:06:23,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:06:23,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:06:23,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:06:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:06:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:13.394619+00:00 (in 48.409083 seconds) repositorygcworker stdout | 2022-05-05 10:06:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:24 UTC)" (scheduled at 2022-05-05 10:06:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:06:24,986 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:06:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 24, 986036), True, datetime.datetime(2022, 5, 5, 10, 6, 24, 986036), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:06:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:06:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:06:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:06:25,121 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:06:25,754 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:06:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:06:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:56.865082+00:00 (in 29.999511 seconds) gcworker stdout | 2022-05-05 10:06:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:56 UTC)" (scheduled at 2022-05-05 10:06:26.865082+00:00) gcworker stdout | 2022-05-05 10:06:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535586866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:06:26,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:06:26,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:06:26,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:06:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:06:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:58.201650+00:00 (in 29.999531 seconds) securityworker stdout | 2022-05-05 10:06:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:58 UTC)" (scheduled at 2022-05-05 10:06:28.201650+00:00) securityworker stdout | 2022-05-05 10:06:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:06:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:06:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:06:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:06:28,219 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:06:28,222 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:06:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:06:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:06:28,226 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 1, 28, 225837), 1, 2]) securityworker stdout | 2022-05-05 10:06:28,228 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:06:28,228 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:06:28,229 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 1, 28, 228786), 1, 2]) securityworker stdout | 2022-05-05 10:06:28,232 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:06:28,232 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:06:28,232 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:06:28,232 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:06:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:06:29,550 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:06:29,722 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:06:30,036 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:06:30,634 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:06:30,936 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:06:31,292 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:06:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:06:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:35.969483+00:00 (in 4.194911 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:06:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:31 UTC)" (scheduled at 2022-05-05 10:06:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:06:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:06:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 31, 775137), True, datetime.datetime(2022, 5, 5, 10, 6, 31, 775137), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:06:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:06:31,786 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:06:31,786 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:06:32,369 [222] [DEBUG] [app] Starting request: urn:request:10a2c3cb-6bde-4771-a7ce-abf07cdd33ce (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:32,369 [219] [DEBUG] [app] Starting request: urn:request:5a6f11fc-3dfb-48e5-9586-76e57cd22a9d (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:32,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:32,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:06:32,374 [231] [DEBUG] [app] Starting request: urn:request:3be28ac1-8482-4bf9-8652-72474466cf20 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:32,374 [224] [DEBUG] [app] Starting request: urn:request:88b02701-6c98-48f6-91eb-89957c376457 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:32,375 [224] [DEBUG] [app] Ending request: urn:request:88b02701-6c98-48f6-91eb-89957c376457 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:32,375 [231] [DEBUG] [app] Ending request: urn:request:3be28ac1-8482-4bf9-8652-72474466cf20 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:06:32,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:06:32,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:32,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:32,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:32,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:32,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:32,379 [219] [DEBUG] [app] Starting request: urn:request:0e30c22a-7b64-4b41-a367-e5cba850f8e8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:32,380 [219] [DEBUG] [app] Ending request: urn:request:0e30c22a-7b64-4b41-a367-e5cba850f8e8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:32,380 [220] [DEBUG] [app] Starting request: urn:request:67eaaaa7-5a27-4898-a5d8-a7fe83a6e6e5 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:06:32,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:32,380 [220] [DEBUG] [app] Ending request: urn:request:67eaaaa7-5a27-4898-a5d8-a7fe83a6e6e5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:32,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:32,381 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:32,381 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:32,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:06:32,382 [222] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:06:32,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:06:32,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:06:32,398 [222] [DEBUG] [app] Ending request: urn:request:10a2c3cb-6bde-4771-a7ce-abf07cdd33ce (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:32,398 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:06:32,398 [219] [DEBUG] [app] Ending request: urn:request:5a6f11fc-3dfb-48e5-9586-76e57cd22a9d (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:32,398 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:06:32,398 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:06:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:06:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.030) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:06:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.031) gunicorn-web stdout | 2022-05-05 10:06:32,399 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:06:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" manifestbackfillworker stdout | 2022-05-05 10:06:32,628 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:06:32,733 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:06:32,894 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:06:32,899 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:06:32,906 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:06:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:06:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:43.089985+00:00 (in 9.999369 seconds) notificationworker stdout | 2022-05-05 10:06:33,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:43 UTC)" (scheduled at 2022-05-05 10:06:33.089985+00:00) notificationworker stdout | 2022-05-05 10:06:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:06:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 33, 91217), True, datetime.datetime(2022, 5, 5, 10, 6, 33, 91217), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:06:33,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:06:33,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:06:33,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:43 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:06:33,206 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:06:33,311 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:06:33,431 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:06:34,195 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:06:35,167 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:06:35,372 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:06:35,474 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:06:35,642 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:06:35,913 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:06:35,919 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:06:35,928 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:06:35,936 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:06:35,945 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:06:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:06:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:31.774027+00:00 (in 55.803949 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:06:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:35 UTC)" (scheduled at 2022-05-05 10:06:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:06:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:06:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:35 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-05 10:06:35,974 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:06:36,821 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:06:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:06:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:14.286471+00:00 (in 35.308022 seconds) namespacegcworker stdout | 2022-05-05 10:06:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:38 UTC)" (scheduled at 2022-05-05 10:06:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:06:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:06:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 38, 978988), True, datetime.datetime(2022, 5, 5, 10, 6, 38, 978988), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:06:38,988 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:06:38,988 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:06:38,989 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:06:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:06:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:06:53.089985+00:00 (in 9.999475 seconds) notificationworker stdout | 2022-05-05 10:06:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:53 UTC)" (scheduled at 2022-05-05 10:06:43.089985+00:00) notificationworker stdout | 2022-05-05 10:06:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:06:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 43, 91053), True, datetime.datetime(2022, 5, 5, 10, 6, 43, 91053), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:06:43,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:06:43,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:06:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:06:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:06:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:06:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:14.479974+00:00 (in 29.999417 seconds) buildlogsarchiver stdout | 2022-05-05 10:06:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:14 UTC)" (scheduled at 2022-05-05 10:06:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:06:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 6, 44, 481000), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:06:44,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:06:44,492 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:06:44,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:06:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:06:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:00.978909+00:00 (in 13.790941 seconds) exportactionlogsworker stdout | 2022-05-05 10:06:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:47 UTC)" (scheduled at 2022-05-05 10:06:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:06:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:06:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:07:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:06:47,370 [221] [DEBUG] [app] Starting request: urn:request:0bdeb8e5-ef1b-4343-b7ae-16c508263cf0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:47,370 [222] [DEBUG] [app] Starting request: urn:request:04eabcdb-5309-4f8d-ae67-4e5e2e257403 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:47,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:47,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:06:47,375 [231] [DEBUG] [app] Starting request: urn:request:035d3c7a-6848-4883-a490-7ef5b2b4186f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:47,375 [230] [DEBUG] [app] Starting request: urn:request:c1b1e111-3bed-4c82-9649-e6ec9bc7a9c3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:47,375 [231] [DEBUG] [app] Ending request: urn:request:035d3c7a-6848-4883-a490-7ef5b2b4186f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:47,376 [230] [DEBUG] [app] Ending request: urn:request:c1b1e111-3bed-4c82-9649-e6ec9bc7a9c3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:06:47,376 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:06:47,376 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:47,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:47,376 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.003) gunicorn-web stdout | 2022-05-05 10:06:47,378 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:47,379 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:06:47,380 [219] [DEBUG] [app] Starting request: urn:request:3b1a7d67-d9b7-407d-90c9-fcccfa9d9948 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:47,381 [219] [DEBUG] [app] Ending request: urn:request:3b1a7d67-d9b7-407d-90c9-fcccfa9d9948 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.002) gunicorn-web stdout | 2022-05-05 10:06:47,382 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:06:47,382 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:47,382 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:06:47,383 [220] [DEBUG] [app] Starting request: urn:request:7e438072-0233-4580-bd6f-7eda3a85ff22 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:47,384 [220] [DEBUG] [app] Ending request: urn:request:7e438072-0233-4580-bd6f-7eda3a85ff22 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:06:47,384 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:06:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:06:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:06:47,385 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:06:47,385 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:06:47,391 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:06:47,392 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:06:47,398 [221] [DEBUG] [app] Ending request: urn:request:0bdeb8e5-ef1b-4343-b7ae-16c508263cf0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:47,398 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:06:47,398 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:06:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:06:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.030) gunicorn-web stdout | 2022-05-05 10:06:47,400 [222] [DEBUG] [app] Ending request: urn:request:04eabcdb-5309-4f8d-ae67-4e5e2e257403 (/health/instance) gunicorn-web stdout | 2022-05-05 10:06:47,400 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:06:47,400 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:06:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:06:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.031) gunicorn-registry stdout | 2022-05-05 10:06:50,787 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:06:50,796 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:06:50,809 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:06:50,813 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:06:50,818 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:06:50,821 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:06:50,828 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:06:50,833 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:06:50,839 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:06:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:06:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:03.089985+00:00 (in 9.999449 seconds) notificationworker stdout | 2022-05-05 10:06:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:03 UTC)" (scheduled at 2022-05-05 10:06:53.089985+00:00) notificationworker stdout | 2022-05-05 10:06:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:06:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 6, 53, 91039), True, datetime.datetime(2022, 5, 5, 10, 6, 53, 91039), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:06:53,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:06:53,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:06:53,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:06:55,162 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:06:55,794 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:06:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:06:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:26.865082+00:00 (in 29.999440 seconds) gcworker stdout | 2022-05-05 10:06:56,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:26 UTC)" (scheduled at 2022-05-05 10:06:56.865082+00:00) gcworker stdout | 2022-05-05 10:06:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535616866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:06:56,879 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:06:56,879 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:06:56,879 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:06:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:06:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:28.201650+00:00 (in 29.999468 seconds) securityworker stdout | 2022-05-05 10:06:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:28 UTC)" (scheduled at 2022-05-05 10:06:58.201650+00:00) securityworker stdout | 2022-05-05 10:06:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:06:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:06:58,207 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:06:58,209 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:06:58,218 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:06:58,219 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:06:59,574 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:06:59,758 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:07:00,056 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:07:00,671 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:07:00,958 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:07:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:07:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:47.187465+00:00 (in 46.207987 seconds) exportactionlogsworker stdout | 2022-05-05 10:07:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:00 UTC)" (scheduled at 2022-05-05 10:07:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:07:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:07:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 0, 980020), True, datetime.datetime(2022, 5, 5, 10, 7, 0, 980020), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:07:00,989 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:07:00,990 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:07:00,990 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:00 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:07:01,335 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:07:02,370 [222] [DEBUG] [app] Starting request: urn:request:d590b6de-da6b-488e-8bb9-9615b376bc8b (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:02,370 [219] [DEBUG] [app] Starting request: urn:request:d60ea8ee-8eb9-40f3-8fc5-0a1956cb7cef (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:02,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:02,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:07:02,374 [229] [DEBUG] [app] Starting request: urn:request:b0d9f36d-6db7-4b40-ae82-29d517038405 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:02,375 [224] [DEBUG] [app] Starting request: urn:request:b79cb4b8-303f-4588-97ea-2f1110cbf4ae (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:02,375 [229] [DEBUG] [app] Ending request: urn:request:b0d9f36d-6db7-4b40-ae82-29d517038405 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:07:02,376 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:07:02,376 [224] [DEBUG] [app] Ending request: urn:request:b79cb4b8-303f-4588-97ea-2f1110cbf4ae (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:02,376 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:02,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:02,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.003) gunicorn-web stdout | 2022-05-05 10:07:02,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:02,379 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:02,380 [221] [DEBUG] [app] Starting request: urn:request:4ccab28f-9378-4616-9273-4b6a0a2baa82 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:02,381 [219] [DEBUG] [app] Starting request: urn:request:f4823cf4-fcc7-4f3b-a86b-c30df3e7605f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:02,381 [221] [DEBUG] [app] Ending request: urn:request:4ccab28f-9378-4616-9273-4b6a0a2baa82 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:07:02,382 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:02,382 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:02,382 [219] [DEBUG] [app] Ending request: urn:request:f4823cf4-fcc7-4f3b-a86b-c30df3e7605f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:02,382 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:07:02,383 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.002) gunicorn-web stdout | 2022-05-05 10:07:02,383 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:02,384 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:07:02,390 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:07:02,392 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:07:02,398 [222] [DEBUG] [app] Ending request: urn:request:d590b6de-da6b-488e-8bb9-9615b376bc8b (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:02,398 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:07:02,398 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:07:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:07:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) gunicorn-web stdout | 2022-05-05 10:07:02,399 [219] [DEBUG] [app] Ending request: urn:request:d60ea8ee-8eb9-40f3-8fc5-0a1956cb7cef (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:02,400 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:07:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.031) gunicorn-web stdout | 2022-05-05 10:07:02,400 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:07:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" manifestbackfillworker stdout | 2022-05-05 10:07:02,665 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:07:02,771 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:07:02,905 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:07:02,909 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:07:02,916 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:07:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:07:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:12.207990+00:00 (in 9.117442 seconds) notificationworker stdout | 2022-05-05 10:07:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:13 UTC)" (scheduled at 2022-05-05 10:07:03.089985+00:00) notificationworker stdout | 2022-05-05 10:07:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:07:03,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 3, 90977), True, datetime.datetime(2022, 5, 5, 10, 7, 3, 90977), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:07:03,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:07:03,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:07:03,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:13 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:07:03,221 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:07:03,347 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:07:03,469 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:07:04,216 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:07:05,207 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:07:05,412 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:07:05,503 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:07:05,678 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:07:05,921 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:07:05,928 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:07:05,936 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:07:05,944 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:07:05,953 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:07:06,010 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:07:06,854 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:07:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:07:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:13.089985+00:00 (in 0.881482 seconds) notificationworker stdout | 2022-05-05 10:07:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:12 UTC)" (scheduled at 2022-05-05 10:07:12.207990+00:00) notificationworker stdout | 2022-05-05 10:07:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:07:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:07:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:07:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:23.089985+00:00 (in 9.999470 seconds) notificationworker stdout | 2022-05-05 10:07:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:23 UTC)" (scheduled at 2022-05-05 10:07:13.089985+00:00) notificationworker stdout | 2022-05-05 10:07:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:07:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 13, 90982), True, datetime.datetime(2022, 5, 5, 10, 7, 13, 90982), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:07:13,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:07:13,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:07:13,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:07:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:07:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:24.984990+00:00 (in 11.589821 seconds) repositorygcworker stdout | 2022-05-05 10:07:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:13 UTC)" (scheduled at 2022-05-05 10:07:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:07:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:07:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:07:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:07:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:38.977973+00:00 (in 24.690898 seconds) namespacegcworker stdout | 2022-05-05 10:07:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:14 UTC)" (scheduled at 2022-05-05 10:07:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:07:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:07:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:07:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:07:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:44.479974+00:00 (in 29.999450 seconds) buildlogsarchiver stdout | 2022-05-05 10:07:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:44 UTC)" (scheduled at 2022-05-05 10:07:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:07:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 7, 14, 480917), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:07:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:07:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:07:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:07:17,370 [221] [DEBUG] [app] Starting request: urn:request:d73c9201-e68a-4c90-a3b8-a9c5cca5b529 (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:17,370 [220] [DEBUG] [app] Starting request: urn:request:32ffbdfa-ccd7-4d88-a88f-711e71d3ad5d (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:17,372 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:17,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:07:17,374 [224] [DEBUG] [app] Starting request: urn:request:34817d36-f8cb-407d-a0d5-d66cb3d34516 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:17,375 [231] [DEBUG] [app] Starting request: urn:request:daf1c443-0eb4-45d4-a0a4-7296e2c0dca5 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:17,375 [224] [DEBUG] [app] Ending request: urn:request:34817d36-f8cb-407d-a0d5-d66cb3d34516 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:17,375 [231] [DEBUG] [app] Ending request: urn:request:daf1c443-0eb4-45d4-a0a4-7296e2c0dca5 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-registry stdout | 2022-05-05 10:07:17,376 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:17,376 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:17,376 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2022-05-05 10:07:17,379 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:17,379 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:07:17,379 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:17,381 [220] [DEBUG] [app] Starting request: urn:request:d473b8ef-a6f6-4759-89d1-81af940acae0 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:17,381 [219] [DEBUG] [app] Starting request: urn:request:966231b0-820d-422c-8968-66b0756c99d2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:17,381 [220] [DEBUG] [app] Ending request: urn:request:d473b8ef-a6f6-4759-89d1-81af940acae0 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:17,381 [219] [DEBUG] [app] Ending request: urn:request:966231b0-820d-422c-8968-66b0756c99d2 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:07:17,382 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:17,382 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:17,382 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:17,382 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:17,383 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:07:17,383 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:07:17,389 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:07:17,389 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:07:17,396 [220] [DEBUG] [app] Ending request: urn:request:32ffbdfa-ccd7-4d88-a88f-711e71d3ad5d (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:17,396 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:07:17,396 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:07:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:07:17,397 [221] [DEBUG] [app] Ending request: urn:request:d73c9201-e68a-4c90-a3b8-a9c5cca5b529 (/health/instance) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:07:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.028) gunicorn-web stdout | 2022-05-05 10:07:17,397 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:07:17,397 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:07:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:07:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-registry stdout | 2022-05-05 10:07:20,798 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:07:20,806 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:07:20,821 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:07:20,827 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:07:20,833 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:07:20,838 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:07:20,845 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:07:20,856 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} gunicorn-registry stdout | 2022-05-05 10:07:20,857 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} notificationworker stdout | 2022-05-05 10:07:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:07:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:33.089985+00:00 (in 9.999520 seconds) notificationworker stdout | 2022-05-05 10:07:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:33 UTC)" (scheduled at 2022-05-05 10:07:23.089985+00:00) notificationworker stdout | 2022-05-05 10:07:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:07:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 23, 90885), True, datetime.datetime(2022, 5, 5, 10, 7, 23, 90885), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:07:23,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:07:23,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:07:23,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:07:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:07:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:00.385766+00:00 (in 35.400130 seconds) repositorygcworker stdout | 2022-05-05 10:07:24,986 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:24 UTC)" (scheduled at 2022-05-05 10:07:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:07:24,986 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:07:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 24, 986288), True, datetime.datetime(2022, 5, 5, 10, 7, 24, 986288), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:07:24,998 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:07:24,998 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:07:24,999 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:07:25,201 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:07:25,834 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:07:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:07:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:56.865082+00:00 (in 29.999343 seconds) gcworker stdout | 2022-05-05 10:07:26,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:56 UTC)" (scheduled at 2022-05-05 10:07:26.865082+00:00) gcworker stdout | 2022-05-05 10:07:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535646866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:07:26,880 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:07:26,880 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:07:26,880 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:07:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:07:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:58.201650+00:00 (in 29.999404 seconds) securityworker stdout | 2022-05-05 10:07:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:58 UTC)" (scheduled at 2022-05-05 10:07:28.201650+00:00) securityworker stdout | 2022-05-05 10:07:28,203 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:07:28,203 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:07:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:07:28,208 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:07:28,218 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:07:28,222 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:07:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:07:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:07:28,227 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 2, 28, 225812), 1, 2]) securityworker stdout | 2022-05-05 10:07:28,230 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:07:28,230 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:07:28,232 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 2, 28, 230952), 1, 2]) securityworker stdout | 2022-05-05 10:07:28,234 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:07:28,234 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:07:28,234 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:07:28,234 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:07:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:07:29,615 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:07:29,771 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:07:30,097 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:07:30,712 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:07:30,998 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:07:31,375 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:07:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:07:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:35.969483+00:00 (in 4.194849 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:07:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:31 UTC)" (scheduled at 2022-05-05 10:07:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:07:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:07:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 31, 775281), True, datetime.datetime(2022, 5, 5, 10, 7, 31, 775281), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:07:31,787 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:07:31,787 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:07:31,787 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:07:32,370 [219] [DEBUG] [app] Starting request: urn:request:6075e8bc-8e9f-4e8e-be2b-9126a7ac2c0c (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:32,370 [222] [DEBUG] [app] Starting request: urn:request:81880471-88d9-4bd3-a534-02af08b2ff07 (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:32,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:32,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:07:32,374 [231] [DEBUG] [app] Starting request: urn:request:f97a0ca2-30f2-4ff1-9431-fb7f0e11b61e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:32,375 [231] [DEBUG] [app] Ending request: urn:request:f97a0ca2-30f2-4ff1-9431-fb7f0e11b61e (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:07:32,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:32,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:07:32,376 [227] [DEBUG] [app] Starting request: urn:request:285629e8-bbb0-4778-b40f-f00576a4de47 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:32,377 [227] [DEBUG] [app] Ending request: urn:request:285629e8-bbb0-4778-b40f-f00576a4de47 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:32,378 [227] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:32,378 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:32,378 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:32,380 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:32,380 [219] [DEBUG] [app] Starting request: urn:request:997e488c-2eda-4d7f-b481-9aefca39089a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:32,380 [219] [DEBUG] [app] Ending request: urn:request:997e488c-2eda-4d7f-b481-9aefca39089a (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.004) gunicorn-web stdout | 2022-05-05 10:07:32,381 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:07:32,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:32,382 [222] [DEBUG] [app] Starting request: urn:request:5853dd12-8f21-4a84-9ca5-299a531267c5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:32,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:07:32,383 [222] [DEBUG] [app] Ending request: urn:request:5853dd12-8f21-4a84-9ca5-299a531267c5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:32,383 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:07:32,383 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:32,384 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:07:32,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:07:32,391 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:07:32,398 [219] [DEBUG] [app] Ending request: urn:request:6075e8bc-8e9f-4e8e-be2b-9126a7ac2c0c (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:32,398 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:07:32,399 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:07:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:07:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.030) gunicorn-web stdout | 2022-05-05 10:07:32,401 [222] [DEBUG] [app] Ending request: urn:request:81880471-88d9-4bd3-a534-02af08b2ff07 (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:32,402 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:07:32,402 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:07:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:07:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.033) manifestbackfillworker stdout | 2022-05-05 10:07:32,702 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:07:32,810 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:07:32,914 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:07:32,918 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:07:32,926 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:07:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:07:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:43.089985+00:00 (in 9.999493 seconds) notificationworker stdout | 2022-05-05 10:07:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:43 UTC)" (scheduled at 2022-05-05 10:07:33.089985+00:00) notificationworker stdout | 2022-05-05 10:07:33,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:07:33,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 33, 90983), True, datetime.datetime(2022, 5, 5, 10, 7, 33, 90983), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:07:33,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:07:33,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:07:33,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:43 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:07:33,255 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:07:33,385 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:07:33,506 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:07:34,242 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:07:35,240 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:07:35,449 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:07:35,513 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:07:35,694 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:07:35,933 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:07:35,939 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:07:35,945 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:07:35,953 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:07:35,963 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:07:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:07:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:03.965762+00:00 (in 27.995726 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:07:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:35 UTC)" (scheduled at 2022-05-05 10:07:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:07:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:07:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:35 UTC)" executed successfully securityscanningnotificationworker stdout | 2022-05-05 10:07:36,029 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:07:36,892 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:07:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:07:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:14.286471+00:00 (in 35.307922 seconds) namespacegcworker stdout | 2022-05-05 10:07:38,979 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:38 UTC)" (scheduled at 2022-05-05 10:07:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:07:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:07:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 38, 979227), True, datetime.datetime(2022, 5, 5, 10, 7, 38, 979227), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:07:38,989 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:07:38,989 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:07:38,990 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:07:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:07:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:07:53.089985+00:00 (in 9.999447 seconds) notificationworker stdout | 2022-05-05 10:07:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:53 UTC)" (scheduled at 2022-05-05 10:07:43.089985+00:00) notificationworker stdout | 2022-05-05 10:07:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:07:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 43, 91175), True, datetime.datetime(2022, 5, 5, 10, 7, 43, 91175), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:07:43,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:07:43,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:07:43,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:07:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:07:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:07:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:14.479974+00:00 (in 29.999496 seconds) buildlogsarchiver stdout | 2022-05-05 10:07:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:14 UTC)" (scheduled at 2022-05-05 10:07:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:07:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 7, 44, 481054), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:07:44,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:07:44,493 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:07:44,493 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:07:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:07:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:00.978909+00:00 (in 13.790908 seconds) exportactionlogsworker stdout | 2022-05-05 10:07:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:47 UTC)" (scheduled at 2022-05-05 10:07:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:07:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:07:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:08:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:07:47,369 [222] [DEBUG] [app] Starting request: urn:request:f4abd2e3-fbde-4b90-ac05-f619966877c7 (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:47,370 [219] [DEBUG] [app] Starting request: urn:request:ae4e3a23-0f47-4b33-8bed-b6b411b677d8 (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:47,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:07:47,373 [229] [DEBUG] [app] Starting request: urn:request:5145749b-9032-4e8c-8379-8e5c890efdd7 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:47,374 [229] [DEBUG] [app] Ending request: urn:request:5145749b-9032-4e8c-8379-8e5c890efdd7 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:47,375 [224] [DEBUG] [app] Starting request: urn:request:d5b0392d-6a38-4768-876c-9666d96206dc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:47,375 [224] [DEBUG] [app] Ending request: urn:request:d5b0392d-6a38-4768-876c-9666d96206dc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:07:47,376 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:07:47,376 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.003) gunicorn-web stdout | 2022-05-05 10:07:47,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:47,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:47,379 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:47,379 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:07:47,381 [219] [DEBUG] [app] Starting request: urn:request:a10e4dde-833d-419c-bae8-79677447bc9e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:47,381 [221] [DEBUG] [app] Starting request: urn:request:82dac1f3-b2e9-4f9f-9d62-f4fd32776a85 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:47,381 [219] [DEBUG] [app] Ending request: urn:request:a10e4dde-833d-419c-bae8-79677447bc9e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:47,381 [221] [DEBUG] [app] Ending request: urn:request:82dac1f3-b2e9-4f9f-9d62-f4fd32776a85 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:07:47,382 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:07:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:07:47,382 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:07:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:07:47,382 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:47,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:07:47,383 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:07:47,383 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:07:47,389 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:07:47,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:07:47,396 [222] [DEBUG] [app] Ending request: urn:request:f4abd2e3-fbde-4b90-ac05-f619966877c7 (/health/instance) gunicorn-web stdout | 2022-05-05 10:07:47,396 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:07:47,397 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:07:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:07:47,397 [219] [DEBUG] [app] Ending request: urn:request:ae4e3a23-0f47-4b33-8bed-b6b411b677d8 (/health/instance) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:07:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.028) gunicorn-web stdout | 2022-05-05 10:07:47,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:07:47,398 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:07:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:07:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) gunicorn-registry stdout | 2022-05-05 10:07:50,806 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:07:50,814 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:07:50,831 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:07:50,838 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:07:50,843 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:07:50,851 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:07:50,855 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:07:50,867 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:07:50,870 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:07:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:07:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:03.089985+00:00 (in 9.999330 seconds) notificationworker stdout | 2022-05-05 10:07:53,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:03 UTC)" (scheduled at 2022-05-05 10:07:53.089985+00:00) notificationworker stdout | 2022-05-05 10:07:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:07:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 7, 53, 91297), True, datetime.datetime(2022, 5, 5, 10, 7, 53, 91297), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:07:53,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:07:53,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:07:53,105 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:07:55,231 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:07:55,867 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:07:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:07:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:26.865082+00:00 (in 29.999370 seconds) gcworker stdout | 2022-05-05 10:07:56,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:26 UTC)" (scheduled at 2022-05-05 10:07:56.865082+00:00) gcworker stdout | 2022-05-05 10:07:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535676866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:07:56,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:07:56,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:07:56,879 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:07:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:07:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:28.201650+00:00 (in 29.999451 seconds) securityworker stdout | 2022-05-05 10:07:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:28 UTC)" (scheduled at 2022-05-05 10:07:58.201650+00:00) securityworker stdout | 2022-05-05 10:07:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:07:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:07:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:07:58,208 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:07:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:07:58,217 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:07:59,651 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:07:59,807 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:08:00,130 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:08:00,386 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:08:00,386 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:13.394619+00:00 (in 13.008276 seconds) repositorygcworker stdout | 2022-05-05 10:08:00,386 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:13:00 UTC)" (scheduled at 2022-05-05 10:08:00.385766+00:00) repositorygcworker stdout | 2022-05-05 10:08:00,387 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 8, 0, 386851), 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:08:00,396 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 8, 0, 386851), True, datetime.datetime(2022, 5, 5, 10, 8, 0, 386851), 0, 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:08:00,399 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 8, 0, 386851), True, datetime.datetime(2022, 5, 5, 10, 8, 0, 386851), 0, 'repositorygc/%', False, datetime.datetime(2022, 5, 5, 10, 8, 0, 386851), 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:08:00,401 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:08:00,401 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:13:00 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:08:00,727 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:08:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:08:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:47.187465+00:00 (in 46.207975 seconds) exportactionlogsworker stdout | 2022-05-05 10:08:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:00 UTC)" (scheduled at 2022-05-05 10:08:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:08:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:08:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 0, 979953), True, datetime.datetime(2022, 5, 5, 10, 8, 0, 979953), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:08:00,990 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:08:00,990 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:08:00,990 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:08:01,035 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:08:01,395 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:08:02,370 [222] [DEBUG] [app] Starting request: urn:request:54752c52-c8e8-4d78-8792-9c5ce2b6b7e1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:02,370 [219] [DEBUG] [app] Starting request: urn:request:c97d79e3-3f40-4917-bda7-658cdeb8b5b9 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:02,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:02,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:08:02,374 [231] [DEBUG] [app] Starting request: urn:request:5388f015-8342-4690-8279-b48cccb5e574 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:02,374 [229] [DEBUG] [app] Starting request: urn:request:8688c65a-2cca-438d-a820-d57f89f06e11 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:02,375 [229] [DEBUG] [app] Ending request: urn:request:8688c65a-2cca-438d-a820-d57f89f06e11 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:02,375 [231] [DEBUG] [app] Ending request: urn:request:5388f015-8342-4690-8279-b48cccb5e574 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:02,376 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:02,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:02,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:02,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:02,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:08:02,375 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:02,379 [222] [DEBUG] [app] Starting request: urn:request:edd86245-fcf2-4d20-b804-893f7d2624a5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:02,380 [220] [DEBUG] [app] Starting request: urn:request:ec579967-6ac0-4cb6-a17e-16d30c24e1e3 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:02,380 [222] [DEBUG] [app] Ending request: urn:request:edd86245-fcf2-4d20-b804-893f7d2624a5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:02,380 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:02,381 [220] [DEBUG] [app] Ending request: urn:request:ec579967-6ac0-4cb6-a17e-16d30c24e1e3 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:02,381 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.003) gunicorn-web stdout | 2022-05-05 10:08:02,382 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:02,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:02,382 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:08:02,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:08:02,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:08:02,389 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:08:02,395 [219] [DEBUG] [app] Ending request: urn:request:c97d79e3-3f40-4917-bda7-658cdeb8b5b9 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:02,395 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:08:02,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:08:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:08:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-web stdout | 2022-05-05 10:08:02,396 [222] [DEBUG] [app] Ending request: urn:request:54752c52-c8e8-4d78-8792-9c5ce2b6b7e1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:02,397 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:08:02,397 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:08:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:08:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) manifestbackfillworker stdout | 2022-05-05 10:08:02,742 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:08:02,846 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:08:02,925 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:08:02,930 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:08:02,937 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:08:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:08:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:12.207990+00:00 (in 9.117416 seconds) notificationworker stdout | 2022-05-05 10:08:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:13 UTC)" (scheduled at 2022-05-05 10:08:03.089985+00:00) notificationworker stdout | 2022-05-05 10:08:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:08:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 3, 91201), True, datetime.datetime(2022, 5, 5, 10, 8, 3, 91201), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:08:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:08:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:08:03,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:13 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:08:03,293 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:08:03,396 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:08:03,545 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} securityscanningnotificationworker stdout | 2022-05-05 10:08:03,966 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:08:03,966 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:31.774027+00:00 (in 27.807754 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:08:03,966 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:13:03 UTC)" (scheduled at 2022-05-05 10:08:03.965762+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:08:03,967 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 8, 3, 966807), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:08:03,979 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 8, 3, 966807), True, datetime.datetime(2022, 5, 5, 10, 8, 3, 966807), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:08:03,983 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 8, 3, 966807), True, datetime.datetime(2022, 5, 5, 10, 8, 3, 966807), 0, 'secscanv4/%', False, datetime.datetime(2022, 5, 5, 10, 8, 3, 966807), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:08:03,986 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:08:03,986 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:13:03 UTC)" executed successfully repositoryactioncounter stdout | 2022-05-05 10:08:04,254 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:08:05,278 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:08:05,487 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:08:05,535 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:08:05,727 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:08:05,942 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:08:05,947 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:08:05,951 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:08:05,961 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:08:05,969 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:08:06,057 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:08:06,927 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:08:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:08:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:13.089985+00:00 (in 0.881368 seconds) notificationworker stdout | 2022-05-05 10:08:12,209 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:12 UTC)" (scheduled at 2022-05-05 10:08:12.207990+00:00) notificationworker stdout | 2022-05-05 10:08:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:08:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:08:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:08:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:23.089985+00:00 (in 9.999367 seconds) notificationworker stdout | 2022-05-05 10:08:13,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:23 UTC)" (scheduled at 2022-05-05 10:08:13.089985+00:00) notificationworker stdout | 2022-05-05 10:08:13,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:08:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 13, 91296), True, datetime.datetime(2022, 5, 5, 10, 8, 13, 91296), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:08:13,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:08:13,105 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:08:13,105 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:08:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:08:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:24.984990+00:00 (in 11.589776 seconds) repositorygcworker stdout | 2022-05-05 10:08:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:13 UTC)" (scheduled at 2022-05-05 10:08:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:08:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:08:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:08:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:08:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:38.977973+00:00 (in 24.690862 seconds) namespacegcworker stdout | 2022-05-05 10:08:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:14 UTC)" (scheduled at 2022-05-05 10:08:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:08:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:08:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:08:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:08:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:44.479974+00:00 (in 29.999367 seconds) buildlogsarchiver stdout | 2022-05-05 10:08:14,481 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:44 UTC)" (scheduled at 2022-05-05 10:08:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:08:14,482 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 8, 14, 481220), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:08:14,497 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:08:14,497 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:08:14,497 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:08:17,370 [221] [DEBUG] [app] Starting request: urn:request:1e8382ba-a466-4dd1-9789-2f8f0f795672 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:17,370 [219] [DEBUG] [app] Starting request: urn:request:19c4a143-8bb0-4d88-bbe7-fca679d112a6 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:17,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:17,374 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:08:17,374 [230] [DEBUG] [app] Starting request: urn:request:17c97c49-6fac-4f9b-8290-d48396f5b6ca (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:17,375 [230] [DEBUG] [app] Ending request: urn:request:17c97c49-6fac-4f9b-8290-d48396f5b6ca (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:17,376 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2022-05-05 10:08:17,379 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:08:17,379 [224] [DEBUG] [app] Starting request: urn:request:bd2b4e2b-29f2-44dc-bd79-83186ff83fc8 (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:17,381 [222] [DEBUG] [app] Starting request: urn:request:e533af77-33e9-4f6c-93f5-0e012258f3f2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:17,381 [222] [DEBUG] [app] Ending request: urn:request:e533af77-33e9-4f6c-93f5-0e012258f3f2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:17,382 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:17,382 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:08:17,383 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-registry stdout | 2022-05-05 10:08:17,384 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:08:17,385 [224] [DEBUG] [app] Ending request: urn:request:bd2b4e2b-29f2-44dc-bd79-83186ff83fc8 (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:17,387 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.009 162 0.008) gunicorn-web stdout | 2022-05-05 10:08:17,389 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:08:17,390 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:17,391 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:08:17,391 [219] [DEBUG] [app] Starting request: urn:request:ed9b59ee-001c-4f9c-bd42-eb149bb3b8ee (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:17,392 [219] [DEBUG] [app] Ending request: urn:request:ed9b59ee-001c-4f9c-bd42-eb149bb3b8ee (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:17,393 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:17,393 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:08:17,394 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:08:17,399 [221] [DEBUG] [app] Ending request: urn:request:1e8382ba-a466-4dd1-9789-2f8f0f795672 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:17,399 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:08:17,399 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:08:17,400 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:08:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:08:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.031) gunicorn-web stdout | 2022-05-05 10:08:17,411 [219] [DEBUG] [app] Ending request: urn:request:19c4a143-8bb0-4d88-bbe7-fca679d112a6 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:17,411 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:08:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.043 118 0.044) gunicorn-web stdout | 2022-05-05 10:08:17,413 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:08:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:08:20,818 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:08:20,832 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:08:20,847 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:08:20,859 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:08:20,865 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:08:20,865 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:08:20,870 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:08:20,880 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:08:20,887 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:08:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:08:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:33.089985+00:00 (in 9.999382 seconds) notificationworker stdout | 2022-05-05 10:08:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:33 UTC)" (scheduled at 2022-05-05 10:08:23.089985+00:00) notificationworker stdout | 2022-05-05 10:08:23,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:08:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 23, 91110), True, datetime.datetime(2022, 5, 5, 10, 8, 23, 91110), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:08:23,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:08:23,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:08:23,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:08:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:08:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:13.394619+00:00 (in 48.408997 seconds) repositorygcworker stdout | 2022-05-05 10:08:24,986 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:24 UTC)" (scheduled at 2022-05-05 10:08:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:08:24,986 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:08:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 24, 986147), True, datetime.datetime(2022, 5, 5, 10, 8, 24, 986147), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:08:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:08:24,997 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:08:24,997 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:08:25,272 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:08:25,906 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:08:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:08:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:56.865082+00:00 (in 29.999381 seconds) gcworker stdout | 2022-05-05 10:08:26,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:56 UTC)" (scheduled at 2022-05-05 10:08:26.865082+00:00) gcworker stdout | 2022-05-05 10:08:26,868 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535706866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:08:26,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:08:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:08:26,879 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:08:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:08:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:58.201650+00:00 (in 29.999517 seconds) securityworker stdout | 2022-05-05 10:08:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:58 UTC)" (scheduled at 2022-05-05 10:08:28.201650+00:00) securityworker stdout | 2022-05-05 10:08:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:08:28,203 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:08:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:08:28,209 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:08:28,219 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:08:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:08:28,224 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:08:28,224 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:08:28,225 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 3, 28, 224695), 1, 2]) securityworker stdout | 2022-05-05 10:08:28,227 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:08:28,227 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:08:28,228 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 3, 28, 227432), 1, 2]) securityworker stdout | 2022-05-05 10:08:28,230 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:08:28,230 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:08:28,230 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:08:28,230 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:08:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:08:29,695 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:08:29,847 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:08:30,170 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:08:30,758 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:08:31,075 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:08:31,435 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:08:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:08:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:35.969483+00:00 (in 4.194867 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:08:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:31 UTC)" (scheduled at 2022-05-05 10:08:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:08:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:08:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 31, 775310), True, datetime.datetime(2022, 5, 5, 10, 8, 31, 775310), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:08:31,788 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:08:31,788 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:08:31,788 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:08:32,369 [219] [DEBUG] [app] Starting request: urn:request:1c798ed2-2e1a-4aec-b8b4-88960c4ed9ca (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:32,369 [222] [DEBUG] [app] Starting request: urn:request:80ede625-dea2-4361-879d-e7e554e0c356 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:32,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:32,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:08:32,374 [230] [DEBUG] [app] Starting request: urn:request:7126470a-a355-45b7-ac5c-3e80a689a9bc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:32,375 [231] [DEBUG] [app] Starting request: urn:request:c459f7ad-9604-4aea-a46b-1b455da665bc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:32,375 [230] [DEBUG] [app] Ending request: urn:request:7126470a-a355-45b7-ac5c-3e80a689a9bc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:32,375 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:08:32,375 [231] [DEBUG] [app] Ending request: urn:request:c459f7ad-9604-4aea-a46b-1b455da665bc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:32,376 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:32,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.003) gunicorn-web stdout | 2022-05-05 10:08:32,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:32,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:32,379 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:32,381 [219] [DEBUG] [app] Starting request: urn:request:3c199f7d-8290-4d78-a400-7bb3489362d1 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:32,381 [219] [DEBUG] [app] Ending request: urn:request:3c199f7d-8290-4d78-a400-7bb3489362d1 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:32,382 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:32,382 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.002) gunicorn-web stdout | 2022-05-05 10:08:32,383 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:08:32,384 [220] [DEBUG] [app] Starting request: urn:request:d668f92a-a3a2-4060-961c-077cfebd5270 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:32,384 [220] [DEBUG] [app] Ending request: urn:request:d668f92a-a3a2-4060-961c-077cfebd5270 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:32,385 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:08:32,385 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:32,386 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:08:32,389 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:08:32,392 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:08:32,396 [222] [DEBUG] [app] Ending request: urn:request:80ede625-dea2-4361-879d-e7e554e0c356 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:32,397 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:08:32,397 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:08:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:08:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.028) gunicorn-web stdout | 2022-05-05 10:08:32,400 [219] [DEBUG] [app] Ending request: urn:request:1c798ed2-2e1a-4aec-b8b4-88960c4ed9ca (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:32,400 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:08:32,401 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:08:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:08:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.033 118 0.033) manifestbackfillworker stdout | 2022-05-05 10:08:32,781 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:08:32,884 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:08:32,936 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:08:32,942 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:08:32,947 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:08:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:08:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:43.089985+00:00 (in 9.999395 seconds) notificationworker stdout | 2022-05-05 10:08:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:43 UTC)" (scheduled at 2022-05-05 10:08:33.089985+00:00) notificationworker stdout | 2022-05-05 10:08:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:08:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 33, 91185), True, datetime.datetime(2022, 5, 5, 10, 8, 33, 91185), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:08:33,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:08:33,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:08:33,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:43 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:08:33,332 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:08:33,433 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:08:33,586 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:08:34,293 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:08:35,315 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:08:35,524 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:08:35,575 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:08:35,766 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:08:35,950 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:08:35,958 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:08:35,967 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} securityscanningnotificationworker stdout | 2022-05-05 10:08:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:08:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:31.774027+00:00 (in 55.803967 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:08:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:35 UTC)" (scheduled at 2022-05-05 10:08:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:08:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:08:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:35 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:08:35,971 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:08:35,979 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:08:36,094 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:08:36,967 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:08:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:08:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:14.286471+00:00 (in 35.307907 seconds) namespacegcworker stdout | 2022-05-05 10:08:38,979 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:38 UTC)" (scheduled at 2022-05-05 10:08:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:08:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:08:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 38, 979200), True, datetime.datetime(2022, 5, 5, 10, 8, 38, 979200), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:08:38,991 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:08:38,991 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:08:38,991 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:08:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:08:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:08:53.089985+00:00 (in 9.999464 seconds) notificationworker stdout | 2022-05-05 10:08:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:53 UTC)" (scheduled at 2022-05-05 10:08:43.089985+00:00) notificationworker stdout | 2022-05-05 10:08:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:08:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 43, 91067), True, datetime.datetime(2022, 5, 5, 10, 8, 43, 91067), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:08:43,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:08:43,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:08:43,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:08:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:08:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:08:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:14.479974+00:00 (in 29.999327 seconds) buildlogsarchiver stdout | 2022-05-05 10:08:44,481 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:14 UTC)" (scheduled at 2022-05-05 10:08:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:08:44,482 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 8, 44, 481271), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:08:44,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:08:44,492 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:08:44,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:08:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:08:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:00.978909+00:00 (in 13.790871 seconds) exportactionlogsworker stdout | 2022-05-05 10:08:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:47 UTC)" (scheduled at 2022-05-05 10:08:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:08:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:08:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:09:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:08:47,369 [221] [DEBUG] [app] Starting request: urn:request:985c5b54-3b8a-4637-b8c6-8961b30a4ee2 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:47,369 [222] [DEBUG] [app] Starting request: urn:request:6d63f06a-957e-4839-beb8-5a8cebe5bb30 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:47,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:47,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:08:47,374 [224] [DEBUG] [app] Starting request: urn:request:2ac6aaee-5c90-4f4f-859b-d9174da9fe73 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:47,374 [227] [DEBUG] [app] Starting request: urn:request:b2f72455-cbf7-47e8-87d4-e923d87e76ce (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:47,374 [224] [DEBUG] [app] Ending request: urn:request:2ac6aaee-5c90-4f4f-859b-d9174da9fe73 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:47,374 [227] [DEBUG] [app] Ending request: urn:request:b2f72455-cbf7-47e8-87d4-e923d87e76ce (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:08:47,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:08:47,375 [227] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:08:47,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:47,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:47,378 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:47,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:08:47,379 [221] [DEBUG] [app] Starting request: urn:request:dd4e2bff-4642-4dea-8bef-e9d42f8e7a7d (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:47,380 [219] [DEBUG] [app] Starting request: urn:request:b877bca8-3785-4ba0-bdcf-1eaf43999545 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:47,380 [221] [DEBUG] [app] Ending request: urn:request:dd4e2bff-4642-4dea-8bef-e9d42f8e7a7d (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:08:47,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:47,380 [219] [DEBUG] [app] Ending request: urn:request:b877bca8-3785-4ba0-bdcf-1eaf43999545 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:08:47,381 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:08:47,381 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:08:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:08:47,381 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:08:47,381 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:08:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:08:47,382 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:08:47,388 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:08:47,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:08:47,396 [221] [DEBUG] [app] Ending request: urn:request:985c5b54-3b8a-4637-b8c6-8961b30a4ee2 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:47,396 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:08:47,396 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:08:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:08:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-web stdout | 2022-05-05 10:08:47,397 [222] [DEBUG] [app] Ending request: urn:request:6d63f06a-957e-4839-beb8-5a8cebe5bb30 (/health/instance) gunicorn-web stdout | 2022-05-05 10:08:47,397 [222] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:08:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) gunicorn-web stdout | 2022-05-05 10:08:47,397 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:08:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:08:50,830 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:08:50,842 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:08:50,855 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:08:50,868 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:08:50,873 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:08:50,878 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:08:50,881 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:08:50,891 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:08:50,899 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:08:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:08:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:03.089985+00:00 (in 9.999471 seconds) notificationworker stdout | 2022-05-05 10:08:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:03 UTC)" (scheduled at 2022-05-05 10:08:53.089985+00:00) notificationworker stdout | 2022-05-05 10:08:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:08:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 8, 53, 91007), True, datetime.datetime(2022, 5, 5, 10, 8, 53, 91007), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:08:53,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:08:53,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:08:53,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:08:55,310 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:08:55,918 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:08:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:08:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:26.865082+00:00 (in 29.999465 seconds) gcworker stdout | 2022-05-05 10:08:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:26 UTC)" (scheduled at 2022-05-05 10:08:56.865082+00:00) gcworker stdout | 2022-05-05 10:08:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535736866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:08:56,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:08:56,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:08:56,879 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:08:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:08:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:28.201650+00:00 (in 29.999433 seconds) securityworker stdout | 2022-05-05 10:08:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:28 UTC)" (scheduled at 2022-05-05 10:08:58.201650+00:00) securityworker stdout | 2022-05-05 10:08:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:08:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:08:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:08:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:08:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:08:58,216 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:08:59,733 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:08:59,885 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:09:00,198 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:09:00,795 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:09:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:09:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:47.187465+00:00 (in 46.208010 seconds) exportactionlogsworker stdout | 2022-05-05 10:09:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:00 UTC)" (scheduled at 2022-05-05 10:09:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:09:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:09:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 0, 980008), True, datetime.datetime(2022, 5, 5, 10, 9, 0, 980008), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:09:00,993 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:09:00,993 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:09:00,993 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:09:01,112 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:09:01,456 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:09:02,370 [221] [DEBUG] [app] Starting request: urn:request:17b6fb47-27fd-4108-83bc-0135e115c9da (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:02,370 [219] [DEBUG] [app] Starting request: urn:request:83704217-48fa-4797-a805-99217929dbe7 (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:02,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:02,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:09:02,373 [227] [DEBUG] [app] Starting request: urn:request:59baa9a2-bf45-4192-919a-8eeaba879294 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:02,374 [227] [DEBUG] [app] Ending request: urn:request:59baa9a2-bf45-4192-919a-8eeaba879294 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:02,374 [227] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:09:02,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:02,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:09:02,377 [224] [DEBUG] [app] Starting request: urn:request:73b0e7e2-35db-4cd5-8c63-a252e0d3db43 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:02,378 [224] [DEBUG] [app] Ending request: urn:request:73b0e7e2-35db-4cd5-8c63-a252e0d3db43 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:02,378 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:09:02,378 [220] [DEBUG] [app] Starting request: urn:request:ec191abe-7f32-438e-851f-f848ac2335fb (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:02,378 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.004) gunicorn-web stdout | 2022-05-05 10:09:02,379 [220] [DEBUG] [app] Ending request: urn:request:ec191abe-7f32-438e-851f-f848ac2335fb (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:02,379 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:09:02,381 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:02,382 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:02,382 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:09:02,383 [222] [DEBUG] [app] Starting request: urn:request:8591bf06-7f1e-4348-80e9-5d5c3d15031f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:02,384 [222] [DEBUG] [app] Ending request: urn:request:8591bf06-7f1e-4348-80e9-5d5c3d15031f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:02,384 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:09:02,385 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:02,385 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:09:02,389 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:09:02,391 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:09:02,397 [221] [DEBUG] [app] Ending request: urn:request:17b6fb47-27fd-4108-83bc-0135e115c9da (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:02,397 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:09:02,397 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:09:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:09:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-web stdout | 2022-05-05 10:09:02,397 [219] [DEBUG] [app] Ending request: urn:request:83704217-48fa-4797-a805-99217929dbe7 (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:02,398 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:09:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) gunicorn-web stdout | 2022-05-05 10:09:02,398 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:09:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" manifestbackfillworker stdout | 2022-05-05 10:09:02,816 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:09:02,895 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:09:02,945 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:09:02,950 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:09:02,955 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:09:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:09:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:12.207990+00:00 (in 9.117503 seconds) notificationworker stdout | 2022-05-05 10:09:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:13 UTC)" (scheduled at 2022-05-05 10:09:03.089985+00:00) notificationworker stdout | 2022-05-05 10:09:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:09:03,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 3, 90981), True, datetime.datetime(2022, 5, 5, 10, 9, 3, 90981), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:09:03,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:09:03,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:09:03,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:13 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:09:03,357 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:09:03,447 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:09:03,620 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:09:04,333 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:09:05,331 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:09:05,560 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:09:05,614 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:09:05,775 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:09:05,960 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:09:05,967 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:09:05,974 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:09:05,979 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:09:05,986 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:09:06,125 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:09:06,996 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:09:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:09:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:13.089985+00:00 (in 0.881504 seconds) notificationworker stdout | 2022-05-05 10:09:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:12 UTC)" (scheduled at 2022-05-05 10:09:12.207990+00:00) notificationworker stdout | 2022-05-05 10:09:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:09:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:09:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:09:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:23.089985+00:00 (in 9.999493 seconds) notificationworker stdout | 2022-05-05 10:09:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:23 UTC)" (scheduled at 2022-05-05 10:09:13.089985+00:00) notificationworker stdout | 2022-05-05 10:09:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:09:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 13, 90973), True, datetime.datetime(2022, 5, 5, 10, 9, 13, 90973), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:09:13,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:09:13,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:09:13,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:09:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:09:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:24.984990+00:00 (in 11.589785 seconds) repositorygcworker stdout | 2022-05-05 10:09:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:13 UTC)" (scheduled at 2022-05-05 10:09:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:09:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:09:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:09:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:09:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:23.280939+00:00 (in 8.993850 seconds) namespacegcworker stdout | 2022-05-05 10:09:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:14 UTC)" (scheduled at 2022-05-05 10:09:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:09:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:09:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:09:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:09:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:44.479974+00:00 (in 29.999408 seconds) buildlogsarchiver stdout | 2022-05-05 10:09:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:44 UTC)" (scheduled at 2022-05-05 10:09:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:09:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 9, 14, 481041), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:09:14,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:09:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:09:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:09:17,369 [219] [DEBUG] [app] Starting request: urn:request:e055bcc3-856e-4527-a137-fb011c93030b (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:17,369 [221] [DEBUG] [app] Starting request: urn:request:f084049a-e786-4739-aa88-dc8f6bb60838 (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:17,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:17,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:09:17,373 [230] [DEBUG] [app] Starting request: urn:request:3baf1144-8003-4788-9d18-0c8246ea07a3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:17,373 [229] [DEBUG] [app] Starting request: urn:request:0fd8d0cd-501b-442b-998b-c84ebad9d4c6 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:17,373 [230] [DEBUG] [app] Ending request: urn:request:3baf1144-8003-4788-9d18-0c8246ea07a3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:17,374 [229] [DEBUG] [app] Ending request: urn:request:0fd8d0cd-501b-442b-998b-c84ebad9d4c6 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:17,374 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:09:17,374 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:09:17,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:17,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:09:17,375 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:17,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:17,377 [219] [DEBUG] [app] Starting request: urn:request:59458b66-4c23-4420-9c49-55561c4a1694 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:17,378 [219] [DEBUG] [app] Ending request: urn:request:59458b66-4c23-4420-9c49-55561c4a1694 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:09:17,378 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:09:17,378 [222] [DEBUG] [app] Starting request: urn:request:ebb8a292-3ccb-4c37-ad97-952f97f9c0b6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:17,378 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:17,379 [222] [DEBUG] [app] Ending request: urn:request:ebb8a292-3ccb-4c37-ad97-952f97f9c0b6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:17,379 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:09:17,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:09:17,380 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:17,381 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:09:17,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:09:17,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:09:17,394 [219] [DEBUG] [app] Ending request: urn:request:e055bcc3-856e-4527-a137-fb011c93030b (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:17,394 [221] [DEBUG] [app] Ending request: urn:request:f084049a-e786-4739-aa88-dc8f6bb60838 (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:17,394 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:09:17,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:09:17,394 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:09:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:09:17,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:09:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:09:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.025) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:09:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.025) gunicorn-registry stdout | 2022-05-05 10:09:20,840 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:09:20,854 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:09:20,864 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:09:20,880 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:09:20,884 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:09:20,888 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:09:20,891 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:09:20,898 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:09:20,907 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:09:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:09:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:33.089985+00:00 (in 9.999501 seconds) notificationworker stdout | 2022-05-05 10:09:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:33 UTC)" (scheduled at 2022-05-05 10:09:23.089985+00:00) notificationworker stdout | 2022-05-05 10:09:23,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:09:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 23, 91039), True, datetime.datetime(2022, 5, 5, 10, 9, 23, 91039), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:09:23,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:09:23,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:09:23,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:33 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:09:23,281 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:09:23,281 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:38.977973+00:00 (in 15.696553 seconds) namespacegcworker stdout | 2022-05-05 10:09:23,281 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:14:23 UTC)" (scheduled at 2022-05-05 10:09:23.280939+00:00) namespacegcworker stdout | 2022-05-05 10:09:23,282 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 9, 23, 281941), 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:09:23,294 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 9, 23, 281941), True, datetime.datetime(2022, 5, 5, 10, 9, 23, 281941), 0, 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:09:23,297 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 9, 23, 281941), True, datetime.datetime(2022, 5, 5, 10, 9, 23, 281941), 0, 'namespacegc/%', False, datetime.datetime(2022, 5, 5, 10, 9, 23, 281941), 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:09:23,299 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:09:23,299 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:14:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:09:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:09:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:13.394619+00:00 (in 48.409090 seconds) repositorygcworker stdout | 2022-05-05 10:09:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:24 UTC)" (scheduled at 2022-05-05 10:09:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:09:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:09:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 24, 986005), True, datetime.datetime(2022, 5, 5, 10, 9, 24, 986005), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:09:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:09:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:09:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:09:25,347 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:09:25,955 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:09:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:09:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:56.865082+00:00 (in 29.999502 seconds) gcworker stdout | 2022-05-05 10:09:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:56 UTC)" (scheduled at 2022-05-05 10:09:26.865082+00:00) gcworker stdout | 2022-05-05 10:09:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535766866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:09:26,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:09:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:09:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:09:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:09:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:58.201650+00:00 (in 29.999480 seconds) securityworker stdout | 2022-05-05 10:09:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:58 UTC)" (scheduled at 2022-05-05 10:09:28.201650+00:00) securityworker stdout | 2022-05-05 10:09:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:09:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:09:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:09:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:09:28,216 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:09:28,219 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:09:28,222 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:09:28,222 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:09:28,222 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 4, 28, 222301), 1, 2]) securityworker stdout | 2022-05-05 10:09:28,224 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:09:28,224 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:09:28,225 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 4, 28, 225086), 1, 2]) securityworker stdout | 2022-05-05 10:09:28,228 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:09:28,228 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:09:28,228 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:09:28,228 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:09:58 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:09:29,074 [222] [DEBUG] [app] Starting request: urn:request:f2babf87-9f3b-4c86-9b38-c62c4848e1f2 (/api/v1/user/notifications) gunicorn-web stdout | 2022-05-05 10:09:29,074 [222] [DEBUG] [app] User loader loading deferred user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:09:29,075 [222] [DEBUG] [auth.cookie] Loading user from cookie: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:09:29,075 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['105a42ac-0110-4aed-8579-1c532997244a', False, 1, 0]) gunicorn-web stdout | 2022-05-05 10:09:29,088 [222] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-05 10:09:29,088 [222] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-05 10:09:29,088 [222] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:09:29,088 [222] [DEBUG] [endpoints.api] Checking permission for user ibazulic gunicorn-web stdout | 2022-05-05 10:09:29,088 [222] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:09:29,089 [222] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:09:29,089 [222] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:09:29,089 [222] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:09:29,091 [222] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2022-05-05 10:09:29,096 [221] [DEBUG] [app] Starting request: urn:request:8820d880-7daf-41ea-8c2f-45b00119cb0e (/api/v1/user/apptoken) gunicorn-web stdout | 2022-05-05 10:09:29,096 [221] [DEBUG] [app] User loader loading deferred user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:09:29,097 [221] [DEBUG] [auth.cookie] Loading user from cookie: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:09:29,097 [222] [DEBUG] [app] Ending request: urn:request:f2babf87-9f3b-4c86-9b38-c62c4848e1f2 (/api/v1/user/notifications) gunicorn-web stdout | 2022-05-05 10:09:29,098 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['105a42ac-0110-4aed-8579-1c532997244a', False, 1, 0]) gunicorn-web stdout | 2022-05-05 10:09:29,098 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:09:29,099 [222] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:09:29 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:09:29 +0000] "GET /api/v1/user/notifications HTTP/1.1" 200 43 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" (0.025 1294 0.026) gunicorn-web stdout | 2022-05-05 10:09:29,105 [221] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-05 10:09:29,106 [221] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-05 10:09:29,106 [221] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:09:29,106 [221] [DEBUG] [endpoints.api] Checking permission for user ibazulic gunicorn-web stdout | 2022-05-05 10:09:29,106 [221] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:09:29,106 [221] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:09:29,106 [221] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:09:29,107 [221] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:09:29,108 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2022, 5, 8, 5, 21, 29, 107475), datetime.datetime(2022, 5, 5, 10, 9, 29, 107567)]) gunicorn-web stdout | 2022-05-05 10:09:29,111 [221] [DEBUG] [app] Ending request: urn:request:8820d880-7daf-41ea-8c2f-45b00119cb0e (/api/v1/user/apptoken) gunicorn-web stdout | 2022-05-05 10:09:29,112 [221] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.128.2.6 () - - [05/May/2022:10:09:29 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.1" 200 38 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" (0.017 1303 0.018) gunicorn-web stdout | 2022-05-05 10:09:29,112 [221] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:09:29 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" gcworker stdout | 2022-05-05 10:09:29,759 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:09:29,895 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:09:30,228 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:09:30,812 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:09:31,140 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:09:31,490 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:09:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:09:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:35.969483+00:00 (in 4.194920 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:09:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:31 UTC)" (scheduled at 2022-05-05 10:09:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:09:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:09:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 31, 775073), True, datetime.datetime(2022, 5, 5, 10, 9, 31, 775073), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:09:31,785 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:09:31,785 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:09:31,785 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:09:32,369 [219] [DEBUG] [app] Starting request: urn:request:3baca3b2-fe5c-420a-946c-bdedddd4915c (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:32,369 [221] [DEBUG] [app] Starting request: urn:request:22824947-919e-4042-aee3-34b234eb5ae0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:32,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:32,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:09:32,373 [229] [DEBUG] [app] Starting request: urn:request:d9def53e-863d-46fe-bcc0-b83c5357a597 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:32,373 [224] [DEBUG] [app] Starting request: urn:request:a72199c1-a895-4ff2-a6f5-b87107268743 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:32,373 [224] [DEBUG] [app] Ending request: urn:request:a72199c1-a895-4ff2-a6f5-b87107268743 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:32,373 [229] [DEBUG] [app] Ending request: urn:request:d9def53e-863d-46fe-bcc0-b83c5357a597 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:32,374 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:09:32,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:09:32,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:32,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:32,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:32,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:32,377 [222] [DEBUG] [app] Starting request: urn:request:68582a69-942d-4b1c-bbd4-0262b385b2e0 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:32,377 [220] [DEBUG] [app] Starting request: urn:request:f364f7dc-1e89-474f-9c8f-b7d079432d1f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:32,378 [222] [DEBUG] [app] Ending request: urn:request:68582a69-942d-4b1c-bbd4-0262b385b2e0 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:32,378 [220] [DEBUG] [app] Ending request: urn:request:f364f7dc-1e89-474f-9c8f-b7d079432d1f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:32,378 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:09:32,378 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:32,379 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:09:32,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:09:32,379 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:09:32,379 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:09:32,385 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:09:32,385 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:09:32,392 [219] [DEBUG] [app] Ending request: urn:request:3baca3b2-fe5c-420a-946c-bdedddd4915c (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:32,392 [221] [DEBUG] [app] Ending request: urn:request:22824947-919e-4042-aee3-34b234eb5ae0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:32,392 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:09:32,392 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:09:32,392 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:09:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:09:32,392 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:09:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:09:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.024 118 0.024) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:09:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.023 118 0.024) manifestbackfillworker stdout | 2022-05-05 10:09:32,852 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:09:32,923 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:09:32,952 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:09:32,958 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:09:32,963 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:09:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:09:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:43.089985+00:00 (in 9.999341 seconds) notificationworker stdout | 2022-05-05 10:09:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:43 UTC)" (scheduled at 2022-05-05 10:09:33.089985+00:00) notificationworker stdout | 2022-05-05 10:09:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:09:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 33, 91149), True, datetime.datetime(2022, 5, 5, 10, 9, 33, 91149), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:09:33,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:09:33,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:09:33,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:43 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:09:33,373 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:09:33,483 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:09:33,631 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:09:34,343 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:09:35,345 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:09:35,596 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:09:35,621 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:09:35,805 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:09:35,968 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} securityscanningnotificationworker stdout | 2022-05-05 10:09:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:09:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:31.774027+00:00 (in 55.804055 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:09:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:35 UTC)" (scheduled at 2022-05-05 10:09:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:09:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:09:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:35 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:09:35,977 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:09:35,984 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:09:35,987 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:09:35,994 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:09:36,162 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:09:37,023 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:09:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:09:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:14.286471+00:00 (in 35.307924 seconds) namespacegcworker stdout | 2022-05-05 10:09:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:38 UTC)" (scheduled at 2022-05-05 10:09:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:09:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:09:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 38, 979035), True, datetime.datetime(2022, 5, 5, 10, 9, 38, 979035), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:09:38,994 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:09:38,994 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:09:38,994 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:09:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:09:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:09:53.089985+00:00 (in 9.999509 seconds) notificationworker stdout | 2022-05-05 10:09:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:53 UTC)" (scheduled at 2022-05-05 10:09:43.089985+00:00) notificationworker stdout | 2022-05-05 10:09:43,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:09:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 43, 90969), True, datetime.datetime(2022, 5, 5, 10, 9, 43, 90969), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:09:43,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:09:43,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:09:43,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:09:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:09:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:09:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:14.479974+00:00 (in 29.999482 seconds) buildlogsarchiver stdout | 2022-05-05 10:09:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:14 UTC)" (scheduled at 2022-05-05 10:09:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:09:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 9, 44, 481023), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:09:44,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:09:44,490 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:09:44,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:09:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:09:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:00.978909+00:00 (in 13.790957 seconds) exportactionlogsworker stdout | 2022-05-05 10:09:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:47 UTC)" (scheduled at 2022-05-05 10:09:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:09:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:09:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:10:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:09:47,369 [222] [DEBUG] [app] Starting request: urn:request:cdeb3ba0-82ec-42b0-b590-9c820e111bea (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:47,369 [219] [DEBUG] [app] Starting request: urn:request:288ab85a-0a85-459f-8b26-98b57d837c12 (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:47,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:09:47,373 [231] [DEBUG] [app] Starting request: urn:request:145a2564-d4c4-4ba2-bdee-1df4d80e0221 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:47,373 [224] [DEBUG] [app] Starting request: urn:request:75b36791-49a5-4cd7-853c-5dabca292ca4 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:47,373 [231] [DEBUG] [app] Ending request: urn:request:145a2564-d4c4-4ba2-bdee-1df4d80e0221 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:09:47,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:09:47,374 [224] [DEBUG] [app] Ending request: urn:request:75b36791-49a5-4cd7-853c-5dabca292ca4 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:09:47,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:09:47,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-web stdout | 2022-05-05 10:09:47,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:47,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:47,378 [221] [DEBUG] [app] Starting request: urn:request:d74ad931-d9cd-4624-ba27-551122767534 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:47,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:09:47,379 [221] [DEBUG] [app] Ending request: urn:request:d74ad931-d9cd-4624-ba27-551122767534 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:47,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:09:47,380 [219] [DEBUG] [app] Starting request: urn:request:27f9fd05-e7b8-4406-9284-f09563fb39fe (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:47,380 [219] [DEBUG] [app] Ending request: urn:request:27f9fd05-e7b8-4406-9284-f09563fb39fe (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:09:47,381 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:09:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:09:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:09:47,381 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:47,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:09:47,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:09:47,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:09:47,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:09:47,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:09:47,394 [222] [DEBUG] [app] Ending request: urn:request:cdeb3ba0-82ec-42b0-b590-9c820e111bea (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:47,395 [219] [DEBUG] [app] Ending request: urn:request:288ab85a-0a85-459f-8b26-98b57d837c12 (/health/instance) gunicorn-web stdout | 2022-05-05 10:09:47,395 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:09:47,395 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:09:47,395 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:09:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:09:47,395 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:09:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:09:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.026) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:09:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.026) gunicorn-registry stdout | 2022-05-05 10:09:50,851 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:09:50,864 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:09:50,876 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:09:50,891 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:09:50,895 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:09:50,900 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:09:50,903 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:09:50,909 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:09:50,914 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:09:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:09:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:03.089985+00:00 (in 9.999497 seconds) notificationworker stdout | 2022-05-05 10:09:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:03 UTC)" (scheduled at 2022-05-05 10:09:53.089985+00:00) notificationworker stdout | 2022-05-05 10:09:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:09:53,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 9, 53, 90977), True, datetime.datetime(2022, 5, 5, 10, 9, 53, 90977), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:09:53,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:09:53,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:09:53,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:09:55,377 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:09:55,976 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:09:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:09:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:26.865082+00:00 (in 29.999447 seconds) gcworker stdout | 2022-05-05 10:09:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:26 UTC)" (scheduled at 2022-05-05 10:09:56.865082+00:00) gcworker stdout | 2022-05-05 10:09:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535796866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:09:56,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:09:56,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:09:56,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:09:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:09:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:28.201650+00:00 (in 29.999514 seconds) securityworker stdout | 2022-05-05 10:09:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:28 UTC)" (scheduled at 2022-05-05 10:09:58.201650+00:00) securityworker stdout | 2022-05-05 10:09:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:09:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:09:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:09:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:09:58,217 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:09:58,217 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:09:59,778 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:09:59,922 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:10:00,262 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:10:00,823 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:10:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:10:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:47.187465+00:00 (in 46.208001 seconds) exportactionlogsworker stdout | 2022-05-05 10:10:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:00 UTC)" (scheduled at 2022-05-05 10:10:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:10:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:10:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 0, 980016), True, datetime.datetime(2022, 5, 5, 10, 10, 0, 980016), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:10:00,992 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:10:00,992 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:10:00,992 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:10:01,169 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:10:01,526 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:10:02,370 [219] [DEBUG] [app] Starting request: urn:request:7a73ddd5-6cba-411a-ace4-d06e7efd1867 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:02,370 [221] [DEBUG] [app] Starting request: urn:request:9a412c58-a31b-472b-94c4-0ba3f95e3cab (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:02,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:02,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:10:02,373 [224] [DEBUG] [app] Starting request: urn:request:17508561-9acb-49f3-8e9b-99653e2a9554 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:02,373 [231] [DEBUG] [app] Starting request: urn:request:2d927456-d536-48d8-8ee7-c9f90c7a78b0 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:02,374 [224] [DEBUG] [app] Ending request: urn:request:17508561-9acb-49f3-8e9b-99653e2a9554 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:02,374 [231] [DEBUG] [app] Ending request: urn:request:2d927456-d536-48d8-8ee7-c9f90c7a78b0 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:02,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-registry stdout | 2022-05-05 10:10:02,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:02,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:10:02,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:02,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:02,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:02,378 [222] [DEBUG] [app] Starting request: urn:request:408c78f1-a61a-442d-ad3a-f4f4ae261845 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:02,378 [220] [DEBUG] [app] Starting request: urn:request:d5d4ec88-3dbf-4210-9450-382d1df4d72f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:02,379 [222] [DEBUG] [app] Ending request: urn:request:408c78f1-a61a-442d-ad3a-f4f4ae261845 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:02,379 [220] [DEBUG] [app] Ending request: urn:request:d5d4ec88-3dbf-4210-9450-382d1df4d72f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:02,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:10:02,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:02,379 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:02,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:02,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:10:02,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:10:02,386 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:10:02,386 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:10:02,394 [221] [DEBUG] [app] Ending request: urn:request:9a412c58-a31b-472b-94c4-0ba3f95e3cab (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:02,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:10:02,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:10:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:10:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:10:02,395 [219] [DEBUG] [app] Ending request: urn:request:7a73ddd5-6cba-411a-ace4-d06e7efd1867 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:02,395 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:10:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-web stdout | 2022-05-05 10:10:02,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:10:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" manifestbackfillworker stdout | 2022-05-05 10:10:02,867 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:10:02,938 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:10:02,961 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:10:02,966 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:10:02,970 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:10:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:10:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:12.207990+00:00 (in 9.117485 seconds) notificationworker stdout | 2022-05-05 10:10:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:13 UTC)" (scheduled at 2022-05-05 10:10:03.089985+00:00) notificationworker stdout | 2022-05-05 10:10:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:10:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 3, 91009), True, datetime.datetime(2022, 5, 5, 10, 10, 3, 91009), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:10:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:10:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:10:03,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:13 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:10:03,392 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:10:03,520 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:10:03,645 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:10:04,380 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:10:05,370 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:10:05,616 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:10:05,644 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:10:05,842 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:10:05,977 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:10:05,985 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:10:05,992 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:10:05,996 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:10:06,002 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:10:06,170 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:10:07,053 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} gunicorn-registry stdout | 2022-05-05 10:10:11,168 [224] [DEBUG] [app] Starting request: urn:request:378bcc12-ebbf-485d-b7ed-57301b489916 (/v2/) gunicorn-registry stdout | 2022-05-05 10:10:11,168 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {} gunicorn-registry stdout | 2022-05-05 10:10:11,168 [224] [DEBUG] [auth.registry_jwt_auth] No auth header. gunicorn-registry stdout | 2022-05-05 10:10:11,169 [224] [DEBUG] [app] Ending request: urn:request:378bcc12-ebbf-485d-b7ed-57301b489916 (/v2/) gunicorn-registry stdout | 2022-05-05 10:10:11,170 [224] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:11 +0000] "GET /v2/ HTTP/1.1" 401 4 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:11 +0000] "GET /v2/ HTTP/1.1" 401 4 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.002 498 0.003) gunicorn-registry stdout | 2022-05-05 10:10:11,311 [224] [DEBUG] [app] Starting request: urn:request:0e6a0985-2fd9-4b43-921a-1027fe67e1fd (/v2/auth) gunicorn-registry stdout | 2022-05-05 10:10:11,311 [224] [DEBUG] [auth.basic] Attempt to process basic auth header gunicorn-registry stdout | 2022-05-05 10:10:11,313 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['ibazulic', 'ibazulic', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,313 [224] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2022-05-05 10:10:11,322 [224] [DEBUG] [peewee.pool] Created new connection 140384019510832. gunicorn-registry stdout | 2022-05-05 10:10:11,648 [224] [DEBUG] [auth.credentials] Successfully validated credentials for user ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,648 [224] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-registry stdout | 2022-05-05 10:10:11,648 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:11,649 [224] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-registry stdout | 2022-05-05 10:10:11,649 [224] [DEBUG] [endpoints.v2.v2auth] Request audience: quay-quay-quay-enterprise.apps.rcegan.azemea.com gunicorn-registry stdout | 2022-05-05 10:10:11,649 [224] [DEBUG] [endpoints.v2.v2auth] Scope request: ['repository:ibazulic/test-big-layers:push,pull'] gunicorn-registry stdout | 2022-05-05 10:10:11,650 [224] [DEBUG] [endpoints.v2.v2auth] Match: ('ibazulic/test-big-layers', 'ibazulic/test-big-layers', 'push,pull') gunicorn-registry stdout | 2022-05-05 10:10:11,651 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['ibazulic', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,655 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,659 [224] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-registry stdout | 2022-05-05 10:10:11,659 [224] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-registry stdout | 2022-05-05 10:10:11,659 [224] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-registry stdout | 2022-05-05 10:10:11,659 [224] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-registry stdout | 2022-05-05 10:10:11,659 [224] [DEBUG] [endpoints.v2.v2auth] Creating repository: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:11,660 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") INNER JOIN "repositorykind" AS "t3" ON ("t1"."kind_id" = "t3"."id") WHERE (((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) AND ("t3"."name" = %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 'image', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,664 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['ibazulic', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,667 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,670 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:11,671 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:11,673 [224] [DEBUG] [peewee] ('INSERT INTO "repository" ("namespace_user_id", "name", "visibility_id", "description", "badge_token", "kind_id", "trust_enabled", "state") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "repository"."id"', [1, 'test-big-layers', 2, None, '57dd2b87-018f-4843-a685-fed71c3bc0dd', 1, False, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,675 [224] [DEBUG] [peewee] ('INSERT INTO "repositoryactioncount" ("repository_id", "count", "date") VALUES (%s, %s, %s) RETURNING "repositoryactioncount"."id"', [2, 0, datetime.date(2022, 5, 4)]) gunicorn-registry stdout | 2022-05-05 10:10:11,677 [224] [DEBUG] [peewee] ('INSERT INTO "repositorysearchscore" ("repository_id", "score") VALUES (%s, %s) RETURNING "repositorysearchscore"."id"', [2, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,678 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "role" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['admin', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,679 [224] [DEBUG] [peewee] ('INSERT INTO "repositorypermission" ("user_id", "repository_id", "role_id") VALUES (%s, %s, %s) RETURNING "repositorypermission"."id"', [1, 2, 1]) gunicorn-registry stdout | 2022-05-05 10:10:11,683 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,684 [224] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-registry stdout | 2022-05-05 10:10:11,684 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,695 [224] [DEBUG] [app] Ending request: urn:request:0e6a0985-2fd9-4b43-921a-1027fe67e1fd (/v2/auth) gunicorn-registry stdout | 2022-05-05 10:10:11,695 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:11,695 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. nginx stdout | 10.130.2.11 () - ibazulic [05/May/2022:10:10:11 +0000] "GET /v2/auth?account=ibazulic&scope=repository%3Aibazulic%2Ftest-big-layers%3Apush%2Cpull&service=quay-quay-quay-enterprise.apps.rcegan.azemea.com HTTP/1.1" 200 1081 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.385 683 0.386) gunicorn-registry stdout | 2022-05-05 10:10:11,696 [224] [INFO] [gunicorn.access] 10.130.2.11 - ibazulic [05/May/2022:10:10:11 +0000] "GET /v2/auth?account=ibazulic&scope=repository%3Aibazulic%2Ftest-big-layers%3Apush%2Cpull&service=quay-quay-quay-enterprise.apps.rcegan.azemea.com HTTP/1.1" 200 1081 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" gunicorn-registry stdout | 2022-05-05 10:10:11,697 [224] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'push_start', 'namespace': 'ibazulic', 'repository': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:11,834 [224] [DEBUG] [app] Starting request: urn:request:e0ebf582-94ea-4cc1-a101-95366184f358 (/v2/ibazulic/test-big-layers/blobs/sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1) gunicorn-registry stdout | 2022-05-05 10:10:11,834 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:11,834 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,834 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,836 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 10, 11, 835212), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 10, 11, 835247), 'quay', None, datetime.datetime(2022, 5, 4, 10, 10, 11, 835265), datetime.datetime(2022, 4, 28, 10, 10, 11, 835280), None]) gunicorn-registry stdout | 2022-05-05 10:10:11,838 [230] [DEBUG] [app] Starting request: urn:request:cb9e1a23-a1fb-4b38-b22c-f764179e6dde (/v2/ibazulic/test-big-layers/blobs/sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e) gunicorn-registry stdout | 2022-05-05 10:10:11,838 [230] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:11,838 [230] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,838 [230] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,841 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 10, 11, 839685), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 10, 11, 839730), 'quay', None, datetime.datetime(2022, 5, 4, 10, 10, 11, 839752), datetime.datetime(2022, 4, 28, 10, 10, 11, 839768), None]) gunicorn-registry stdout | 2022-05-05 10:10:11,842 [230] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2022-05-05 10:10:11,843 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:11,843 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,843 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,843 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:11,843 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:11,843 [224] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2 gunicorn-registry stdout | 2022-05-05 10:10:11,844 [224] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:11,844 [231] [DEBUG] [app] Starting request: urn:request:a2657b2f-257d-4425-b5a7-d58fad15f1fe (/v2/ibazulic/test-big-layers/blobs/sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a) gunicorn-registry stdout | 2022-05-05 10:10:11,844 [231] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:11,844 [231] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,844 [231] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,845 [229] [DEBUG] [app] Starting request: urn:request:f9509560-56d7-4372-bfb7-e40dcc54684b (/v2/ibazulic/test-big-layers/blobs/sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c) gunicorn-registry stdout | 2022-05-05 10:10:11,845 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,845 [229] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:11,845 [229] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,845 [229] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,847 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 10, 11, 845623), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 10, 11, 845692), 'quay', None, datetime.datetime(2022, 5, 4, 10, 10, 11, 845716), datetime.datetime(2022, 4, 28, 10, 10, 11, 845735), None]) gunicorn-registry stdout | 2022-05-05 10:10:11,847 [231] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2022-05-05 10:10:11,848 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 10, 11, 846635), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 10, 11, 846679), 'quay', None, datetime.datetime(2022, 5, 4, 10, 10, 11, 846698), datetime.datetime(2022, 4, 28, 10, 10, 11, 846710), None]) gunicorn-registry stdout | 2022-05-05 10:10:11,848 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,848 [229] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2022-05-05 10:10:11,849 [230] [DEBUG] [peewee.pool] Created new connection 140384011097648. gunicorn-registry stdout | 2022-05-05 10:10:11,849 [228] [DEBUG] [app] Starting request: urn:request:5b9894ef-06cf-40be-ac97-956367c124e6 (/v2/ibazulic/test-big-layers/blobs/sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1) gunicorn-registry stdout | 2022-05-05 10:10:11,850 [228] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:11,850 [228] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,850 [228] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,851 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,853 [228] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2022, 5, 5, 10, 10, 11, 851448), None, 'quay', 'quay', datetime.datetime(2022, 5, 5, 10, 10, 11, 851479), 'quay', None, datetime.datetime(2022, 5, 4, 10, 10, 11, 851496), datetime.datetime(2022, 4, 28, 10, 10, 11, 851507), None]) gunicorn-registry stdout | 2022-05-05 10:10:11,853 [228] [DEBUG] [peewee.pool] No connection available in pool. gunicorn-registry stdout | 2022-05-05 10:10:11,853 [224] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,853 [224] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,854 [224] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' gunicorn-registry stdout | 2022-05-05 10:10:11,853 [231] [DEBUG] [peewee.pool] Created new connection 140384019515536. gunicorn-registry stdout | 2022-05-05 10:10:11,854 [229] [DEBUG] [peewee.pool] Created new connection 140384010638288. gunicorn-registry stdout | 2022-05-05 10:10:11,855 [224] [DEBUG] [app] Ending request: urn:request:e0ebf582-94ea-4cc1-a101-95366184f358 (/v2/ibazulic/test-big-layers/blobs/sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1) gunicorn-registry stdout | 2022-05-05 10:10:11,855 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:11,855 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.022 1670 0.023) gunicorn-registry stdout | 2022-05-05 10:10:11,855 [224] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" gunicorn-registry stdout | 2022-05-05 10:10:11,857 [230] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:11,857 [230] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,857 [230] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,857 [230] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:11,857 [230] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:11,857 [230] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e_2 gunicorn-registry stdout | 2022-05-05 10:10:11,858 [230] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:11,860 [228] [DEBUG] [peewee.pool] Created new connection 140384011089152. gunicorn-registry stdout | 2022-05-05 10:10:11,860 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,862 [231] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:11,862 [229] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:11,862 [231] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,862 [229] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,862 [231] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,862 [229] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,862 [231] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:11,862 [231] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:11,862 [229] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:11,862 [229] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:11,863 [231] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2 gunicorn-registry stdout | 2022-05-05 10:10:11,863 [229] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c_2 gunicorn-registry stdout | 2022-05-05 10:10:11,863 [231] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:11,863 [229] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:11,865 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:11,865 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,866 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,867 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,868 [228] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:11,868 [228] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,868 [228] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,868 [228] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:11,868 [228] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:11,869 [228] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2 gunicorn-registry stdout | 2022-05-05 10:10:11,869 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:11,869 [228] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:11,870 [230] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,870 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:11,871 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,871 [228] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,872 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,873 [230] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,873 [231] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,874 [229] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,876 [230] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,876 [230] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,876 [228] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "repositorykind" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:11,876 [230] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' gunicorn-registry stdout | 2022-05-05 10:10:11,876 [231] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,877 [230] [DEBUG] [app] Ending request: urn:request:cb9e1a23-a1fb-4b38-b22c-f764179e6dde (/v2/ibazulic/test-big-layers/blobs/sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e) gunicorn-registry stdout | 2022-05-05 10:10:11,877 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:11,877 [230] [DEBUG] [peewee.pool] Returning 140384011097648 to pool. gunicorn-registry stdout | 2022-05-05 10:10:11,877 [229] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,878 [230] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:9d89c9285e5e528fe03a73e17794e9ee2a9146135142c6ae2aa4f35a6a7a132e HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.041 1670 0.041) gunicorn-registry stdout | 2022-05-05 10:10:11,878 [231] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,878 [228] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "visibility" AS "t1" WHERE ("t1"."name" = %s) LIMIT %s OFFSET %s', ['public', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,878 [231] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,879 [231] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' gunicorn-registry stdout | 2022-05-05 10:10:11,880 [229] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,880 [231] [DEBUG] [app] Ending request: urn:request:a2657b2f-257d-4425-b5a7-d58fad15f1fe (/v2/ibazulic/test-big-layers/blobs/sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a) gunicorn-registry stdout | 2022-05-05 10:10:11,880 [229] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,880 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:11,880 [231] [DEBUG] [peewee.pool] Returning 140384019515536 to pool. gunicorn-registry stdout | 2022-05-05 10:10:11,880 [231] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" gunicorn-registry stdout | 2022-05-05 10:10:11,880 [229] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.038 1670 0.037) gunicorn-registry stdout | 2022-05-05 10:10:11,881 [229] [DEBUG] [app] Ending request: urn:request:f9509560-56d7-4372-bfb7-e40dcc54684b (/v2/ibazulic/test-big-layers/blobs/sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c) gunicorn-registry stdout | 2022-05-05 10:10:11,881 [228] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,881 [229] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:11,881 [229] [DEBUG] [peewee.pool] Returning 140384010638288 to pool. gunicorn-registry stdout | 2022-05-05 10:10:11,882 [229] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:273dd1a185869fc9c860e31ed970052b87528d591ac858f93f017d127be3ca9c HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.038 1670 0.038) gunicorn-registry stdout | 2022-05-05 10:10:11,884 [228] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,887 [228] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,887 [228] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2: None gunicorn-registry stdout | 2022-05-05 10:10:11,887 [228] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' gunicorn-registry stdout | 2022-05-05 10:10:11,888 [228] [DEBUG] [app] Ending request: urn:request:5b9894ef-06cf-40be-ac97-956367c124e6 (/v2/ibazulic/test-big-layers/blobs/sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1) gunicorn-registry stdout | 2022-05-05 10:10:11,889 [228] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:11,889 [228] [DEBUG] [peewee.pool] Returning 140384011089152 to pool. gunicorn-registry stdout | 2022-05-05 10:10:11,889 [228] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:11 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.041 1670 0.041) gunicorn-registry stdout | 2022-05-05 10:10:11,991 [224] [DEBUG] [app] Starting request: urn:request:c2e86dad-b04f-4707-87cf-94d6e5e45079 (/v2/ibazulic/test-big-layers/blobs/sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc) gunicorn-registry stdout | 2022-05-05 10:10:11,991 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:11,992 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,992 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:11,993 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:11,993 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,993 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:11,993 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:11,993 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:11,993 [224] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc_2 gunicorn-registry stdout | 2022-05-05 10:10:11,994 [224] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:11,995 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:11,998 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,001 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,003 [224] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc_2: None gunicorn-registry stdout | 2022-05-05 10:10:12,003 [224] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc_2: None gunicorn-registry stdout | 2022-05-05 10:10:12,003 [224] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' gunicorn-registry stdout | 2022-05-05 10:10:12,004 [224] [DEBUG] [app] Ending request: urn:request:c2e86dad-b04f-4707-87cf-94d6e5e45079 (/v2/ibazulic/test-big-layers/blobs/sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc) gunicorn-registry stdout | 2022-05-05 10:10:12,004 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,004 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,004 [224] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:12 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:12 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:04f6ae7e4653529fbcf28a4bda8e722665ad45ef616975d2e8f1a901b7c427fc HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.014 1670 0.013) gunicorn-registry stdout | 2022-05-05 10:10:12,014 [224] [DEBUG] [app] Starting request: urn:request:2505d132-0fd4-4b8a-b2f3-a059508d2aeb (/v2/ibazulic/test-big-layers/blobs/sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4) gunicorn-registry stdout | 2022-05-05 10:10:12,014 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,014 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,014 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,015 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,016 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,016 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,016 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,016 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,016 [224] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4_2 gunicorn-registry stdout | 2022-05-05 10:10:12,016 [224] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:12,017 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,017 [231] [DEBUG] [app] Starting request: urn:request:32904d3f-0191-4180-aea9-19182f1a61f5 (/v2/ibazulic/test-big-layers/blobs/sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9) gunicorn-registry stdout | 2022-05-05 10:10:12,017 [231] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,018 [231] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,018 [231] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,019 [231] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,019 [231] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,019 [231] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,019 [231] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,019 [231] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,019 [231] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9_2 gunicorn-registry stdout | 2022-05-05 10:10:12,019 [231] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:12,020 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,021 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,021 [230] [DEBUG] [app] Starting request: urn:request:651ac414-f9df-47d3-850c-588739b17044 (/v2/ibazulic/test-big-layers/blobs/sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e) gunicorn-registry stdout | 2022-05-05 10:10:12,022 [230] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,022 [230] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,022 [230] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,023 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,023 [230] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,023 [230] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,023 [230] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,023 [230] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,024 [230] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,024 [230] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e_2 gunicorn-registry stdout | 2022-05-05 10:10:12,024 [230] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:12,024 [231] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,025 [224] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4_2: None gunicorn-registry stdout | 2022-05-05 10:10:12,025 [224] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4_2: None gunicorn-registry stdout | 2022-05-05 10:10:12,025 [224] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' gunicorn-registry stdout | 2022-05-05 10:10:12,025 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,026 [224] [DEBUG] [app] Ending request: urn:request:2505d132-0fd4-4b8a-b2f3-a059508d2aeb (/v2/ibazulic/test-big-layers/blobs/sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4) gunicorn-registry stdout | 2022-05-05 10:10:12,026 [231] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,026 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,026 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,026 [224] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:12 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4 HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:12 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:27c726bebd645e5c59753ff7a4075494eeabbcf9c5c95669e5fcdf2bf4f532b4 HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.013 1670 0.014) gunicorn-registry stdout | 2022-05-05 10:10:12,028 [231] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9_2: None gunicorn-registry stdout | 2022-05-05 10:10:12,028 [231] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9_2: None gunicorn-registry stdout | 2022-05-05 10:10:12,028 [230] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,028 [231] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' gunicorn-registry stdout | 2022-05-05 10:10:12,028 [229] [DEBUG] [app] Starting request: urn:request:4ae909f6-a366-4b4d-b6b5-8e9f0f5052f8 (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,028 [229] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,028 [229] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,028 [229] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,029 [231] [DEBUG] [app] Ending request: urn:request:32904d3f-0191-4180-aea9-19182f1a61f5 (/v2/ibazulic/test-big-layers/blobs/sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9) gunicorn-registry stdout | 2022-05-05 10:10:12,029 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,029 [231] [DEBUG] [peewee.pool] Returning 140384019515536 to pool. nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:12 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9 HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.013 1670 0.014) gunicorn-registry stdout | 2022-05-05 10:10:12,029 [231] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:12 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:0983e930d0d0704b96c6a5aa8f46b6f33d451cc8b602cbca620cea8db6bc4cf9 HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" gunicorn-registry stdout | 2022-05-05 10:10:12,030 [229] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,030 [229] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,030 [229] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,030 [229] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,031 [229] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,031 [230] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "manifestblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,032 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,033 [230] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e_2: None gunicorn-registry stdout | 2022-05-05 10:10:12,033 [230] [DEBUG] [data.cache.impl] Not caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e_2: None gunicorn-registry stdout | 2022-05-05 10:10:12,033 [230] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"BLOB_UNKNOWN","detail":{},"message":"blob unknown to registry"}]}\n' gunicorn-registry stdout | 2022-05-05 10:10:12,034 [230] [DEBUG] [app] Ending request: urn:request:651ac414-f9df-47d3-850c-588739b17044 (/v2/ibazulic/test-big-layers/blobs/sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e) gunicorn-registry stdout | 2022-05-05 10:10:12,034 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,034 [230] [DEBUG] [peewee.pool] Returning 140384011097648 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,034 [230] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:12 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:12 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:3e9942704944d6038666a8173a92fa4d7228d5098af5f4724a3ac10cd4171a7e HTTP/1.1" 404 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.014 1670 0.013) gunicorn-registry stdout | 2022-05-05 10:10:12,035 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,037 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:12,040 [229] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [2, '3d53d726-da22-45f9-8594-6669c10e28e1', 0, 'gASVqwAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZRoAYwHbWRfZGF0YZRDeGfmCWqFrme7cvNuPDr1T6V/Ug5RjGgFm6vZgx8ZzeBbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAJR1Yi4=', 9, '{"blocks": [], "content-type": null}', 0, datetime.datetime(2022, 5, 5, 10, 10, 12, 39461)]) gunicorn-registry stdout | 2022-05-05 10:10:12,044 [229] [DEBUG] [app] Ending request: urn:request:4ae909f6-a366-4b4d-b6b5-8e9f0f5052f8 (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,044 [229] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,044 [229] [DEBUG] [peewee.pool] Returning 140384010638288 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,045 [229] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.018 1649 0.017) gunicorn-registry stdout | 2022-05-05 10:10:12,140 [224] [DEBUG] [app] Starting request: urn:request:3a8ae339-9421-497e-af14-3261ab5d785e (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,140 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,140 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,140 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,142 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,142 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,142 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,142 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,142 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,143 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,147 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,149 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:12,152 [224] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [2, 'bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 0, 'gASVqwAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZRoAYwHbWRfZGF0YZRDeGfmCWqFrme7cvNuPDr1T6V/Ug5RjGgFm6vZgx8ZzeBbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAJR1Yi4=', 9, '{"blocks": [], "content-type": null}', 0, datetime.datetime(2022, 5, 5, 10, 10, 12, 151416)]) gunicorn-registry stdout | 2022-05-05 10:10:12,156 [224] [DEBUG] [app] Ending request: urn:request:3a8ae339-9421-497e-af14-3261ab5d785e (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,156 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,156 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,157 [224] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.018 1649 0.018) gunicorn-registry stdout | 2022-05-05 10:10:12,168 [224] [DEBUG] [app] Starting request: urn:request:f0e83eed-b192-47bf-8e8e-fae36dc01dbf (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,168 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,168 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,168 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,170 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,170 [230] [DEBUG] [app] Starting request: urn:request:1821801b-2319-4e8d-8a78-143016fbc2a4 (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,170 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,170 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,170 [230] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,170 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,170 [230] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,170 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,170 [230] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,171 [230] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,171 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,171 [230] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,171 [230] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,171 [230] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,171 [230] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,173 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,174 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,176 [224] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [2, 'a1836993-a72f-466e-acde-71c412576855', 0, 'gASVqwAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZRoAYwHbWRfZGF0YZRDeGfmCWqFrme7cvNuPDr1T6V/Ug5RjGgFm6vZgx8ZzeBbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAJR1Yi4=', 9, '{"blocks": [], "content-type": null}', 0, datetime.datetime(2022, 5, 5, 10, 10, 12, 176360)]) gunicorn-registry stdout | 2022-05-05 10:10:12,178 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,180 [230] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:12,180 [224] [DEBUG] [app] Ending request: urn:request:f0e83eed-b192-47bf-8e8e-fae36dc01dbf (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,181 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,181 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,181 [224] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.014 1649 0.015) gunicorn-registry stdout | 2022-05-05 10:10:12,182 [231] [DEBUG] [app] Starting request: urn:request:91cd96e5-ed9d-424b-b53c-ad93fc18831a (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,182 [231] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,182 [231] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,182 [231] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,183 [230] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [2, '5a78111c-05a1-4b40-9160-2cbb59d6743d', 0, 'gASVqwAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZRoAYwHbWRfZGF0YZRDeGfmCWqFrme7cvNuPDr1T6V/Ug5RjGgFm6vZgx8ZzeBbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAJR1Yi4=', 9, '{"blocks": [], "content-type": null}', 0, datetime.datetime(2022, 5, 5, 10, 10, 12, 182096)]) gunicorn-registry stdout | 2022-05-05 10:10:12,183 [231] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,183 [231] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,183 [231] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,183 [231] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,183 [231] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,184 [224] [DEBUG] [app] Starting request: urn:request:47adc41d-69b2-44a6-baad-988483150a48 (/v2/ibazulic/test-big-layers/blobs/uploads/3d53d726-da22-45f9-8594-6669c10e28e1) gunicorn-registry stdout | 2022-05-05 10:10:12,184 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': '3d53d726-da22-45f9-8594-6669c10e28e1', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,184 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,184 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,185 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,186 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,186 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,186 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,186 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,186 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,187 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,188 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,189 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "imagestoragelocation" AS "t1"', []) gunicorn-registry stdout | 2022-05-05 10:10:12,189 [230] [DEBUG] [app] Ending request: urn:request:1821801b-2319-4e8d-8a78-143016fbc2a4 (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,190 [230] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,190 [230] [DEBUG] [peewee.pool] Returning 140384011097648 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,190 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d53d726-da22-45f9-8594-6669c10e28e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,190 [230] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.022 1649 0.022) gunicorn-registry stdout | 2022-05-05 10:10:12,192 [231] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [2, 'ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 0, 'gASVqwAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZRoAYwHbWRfZGF0YZRDeGfmCWqFrme7cvNuPDr1T6V/Ug5RjGgFm6vZgx8ZzeBbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAJR1Yi4=', 9, '{"blocks": [], "content-type": null}', 0, datetime.datetime(2022, 5, 5, 10, 10, 12, 192211)]) gunicorn-registry stdout | 2022-05-05 10:10:12,193 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,193 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,197 [231] [DEBUG] [app] Ending request: urn:request:91cd96e5-ed9d-424b-b53c-ad93fc18831a (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:10:12,197 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,197 [231] [DEBUG] [peewee.pool] Returning 140384019515536 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,198 [231] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:12 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.017 1649 0.017) notificationworker stdout | 2022-05-05 10:10:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:10:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:13.089985+00:00 (in 0.881500 seconds) notificationworker stdout | 2022-05-05 10:10:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:12 UTC)" (scheduled at 2022-05-05 10:10:12.207990+00:00) notificationworker stdout | 2022-05-05 10:10:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:10:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:12 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:10:12,323 [224] [DEBUG] [app] Starting request: urn:request:aef6f690-d95f-4352-8675-8013ab147bfc (/v2/ibazulic/test-big-layers/blobs/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6) gunicorn-registry stdout | 2022-05-05 10:10:12,323 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': 'bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,323 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,323 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,325 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,325 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,325 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,325 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,325 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,327 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,330 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,332 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,333 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,357 [224] [DEBUG] [app] Starting request: urn:request:0fc935a2-aead-4aaa-bb75-ef141edb1b14 (/v2/ibazulic/test-big-layers/blobs/uploads/a1836993-a72f-466e-acde-71c412576855) gunicorn-registry stdout | 2022-05-05 10:10:12,357 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': 'a1836993-a72f-466e-acde-71c412576855', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,357 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,357 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,359 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,359 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,359 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,359 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,359 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,360 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,364 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['a1836993-a72f-466e-acde-71c412576855', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,366 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,366 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,456 [224] [DEBUG] [app] Starting request: urn:request:a02b4914-ee0b-4a76-8bb4-c4f3dc7a9d80 (/v2/ibazulic/test-big-layers/blobs/uploads/5a78111c-05a1-4b40-9160-2cbb59d6743d) gunicorn-registry stdout | 2022-05-05 10:10:12,456 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': '5a78111c-05a1-4b40-9160-2cbb59d6743d', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,456 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,456 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,458 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,458 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,458 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,458 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,458 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,458 [231] [DEBUG] [app] Starting request: urn:request:959c7215-195d-4228-b25d-c5b0de290e0b (/v2/ibazulic/test-big-layers/blobs/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035) gunicorn-registry stdout | 2022-05-05 10:10:12,459 [231] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': 'ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:12,459 [231] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,459 [231] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:12,460 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,460 [231] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:12,460 [231] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,460 [231] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:12,461 [231] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:12,461 [231] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:12,462 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,464 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['5a78111c-05a1-4b40-9160-2cbb59d6743d', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,465 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,466 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,466 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,467 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,467 [231] [DEBUG] [peewee.pool] Returning 140384019515536 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,500 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/3d53d726-da22-45f9-8594-6669c10e28e1?blockid=REDACTED&comp=REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/octet-stream' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '585146' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8d37fb38-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-MD5': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:12,501 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] A body is sent with the request gunicorn-registry stdout | 2022-05-05 10:10:12,504 [224] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaystorageaccount2.blob.core.windows.net:443 gunicorn-registry stdout | 2022-05-05 10:10:12,715 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/uploads/3d53d726-da22-45f9-8594-6669c10e28e1?blockid=MDAwMDA%3D&comp=block HTTP/1.1" 201 0 gunicorn-registry stdout | 2022-05-05 10:10:12,715 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 201 gunicorn-registry stdout | 2022-05-05 10:10:12,715 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:12,715 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:12,716 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-MD5': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:12,716 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:12,716 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86abee1-401e-0047-7c68-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:12,716 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8d37fb38-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:12,716 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:12,716 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:12,716 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:12 GMT' gunicorn-registry stdout | 2022-05-05 10:10:12,717 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d53d726-da22-45f9-8594-6669c10e28e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:12,720 [224] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [2, '3d53d726-da22-45f9-8594-6669c10e28e1', 585146, 'gASVsgAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZSMBnNoYTI1NpSMB21kX2RhdGGUQ3h2+fxeRfCwdfw3BJhoJdporXm8ziy0rVs/N/5xX9TLqdBtRwAAAAAA1d1r6N1n6czgnGlI3czcHjzIMs/gNy6FwBNYWx9/9oa44bPhs+Gz4fP/yOf/CwAA//+fP0kwAGIXAAAAAAAAADoAAAAgAAAAAAAAAAAAAACUdWIu', 9, '{"blocks": ["00000"], "content-type": "application/octet-stream"}', 1, 1532416, datetime.datetime(2022, 5, 5, 10, 10, 12, 39461), None, None, 9]) gunicorn-registry stdout | 2022-05-05 10:10:12,725 [224] [DEBUG] [app] Ending request: urn:request:47adc41d-69b2-44a6-baad-988483150a48 (/v2/ibazulic/test-big-layers/blobs/uploads/3d53d726-da22-45f9-8594-6669c10e28e1) gunicorn-registry stdout | 2022-05-05 10:10:12,725 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:12,725 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:12,725 [224] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:12 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/3d53d726-da22-45f9-8594-6669c10e28e1 HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:12 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/3d53d726-da22-45f9-8594-6669c10e28e1 HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.543 587112 0.542) notificationworker stdout | 2022-05-05 10:10:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:10:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:23.089985+00:00 (in 9.999346 seconds) notificationworker stdout | 2022-05-05 10:10:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:23 UTC)" (scheduled at 2022-05-05 10:10:13.089985+00:00) notificationworker stdout | 2022-05-05 10:10:13,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:10:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 13, 91130), True, datetime.datetime(2022, 5, 5, 10, 10, 13, 91130), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:10:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:10:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:10:13,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:23 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:10:13,218 [224] [DEBUG] [app] Starting request: urn:request:119f89fd-eed9-4b60-9c85-a79bbccf4815 (/v2/ibazulic/test-big-layers/blobs/uploads/3d53d726-da22-45f9-8594-6669c10e28e1) gunicorn-registry stdout | 2022-05-05 10:10:13,219 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': '3d53d726-da22-45f9-8594-6669c10e28e1', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:13,219 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:13,219 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:13,220 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:13,220 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:13,220 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:13,220 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:13,220 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:13,222 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:13,226 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d53d726-da22-45f9-8594-6669c10e28e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:13,228 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:13,228 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:13,229 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d53d726-da22-45f9-8594-6669c10e28e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:13,232 [224] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [2, '3d53d726-da22-45f9-8594-6669c10e28e1', 585146, 'gASVsgAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZSMBnNoYTI1NpSMB21kX2RhdGGUQ3h2+fxeRfCwdfw3BJhoJdporXm8ziy0rVs/N/5xX9TLqdBtRwAAAAAA1d1r6N1n6czgnGlI3czcHjzIMs/gNy6FwBNYWx9/9oa44bPhs+Gz4fP/yOf/CwAA//+fP0kwAGIXAAAAAAAAADoAAAAgAAAAAAAAAAAAAACUdWIu', 9, '{"blocks": ["00000"], "content-type": "application/octet-stream"}', 2, 1532416, datetime.datetime(2022, 5, 5, 10, 10, 12, 39461), None, None, 9]) gunicorn-registry stdout | 2022-05-05 10:10:13,237 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:13,237 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1' gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'HEAD' gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8da8c138-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,238 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:13,260 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "HEAD /quayblobstorage/registry/sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 404 0 gunicorn-registry stdout | 2022-05-05 10:10:13,260 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 404 gunicorn-registry stdout | 2022-05-05 10:10:13,260 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:13,261 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Transfer-Encoding': 'chunked' gunicorn-registry stdout | 2022-05-05 10:10:13,261 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:13,261 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86ac17e-401e-0047-7368-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:13,261 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8da8c138-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,261 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,261 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-error-code': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,261 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:12 GMT' gunicorn-registry stdout | 2022-05-05 10:10:13,263 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/3d53d726-da22-45f9-8594-6669c10e28e1?comp=REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,263 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:13,263 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:13,263 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/xml; charset=utf-8' gunicorn-registry stdout | 2022-05-05 10:10:13,263 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,263 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-blob-content-type': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,263 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '87' gunicorn-registry stdout | 2022-05-05 10:10:13,263 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,264 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8dac986c-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,264 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:13,264 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,264 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] A body is sent with the request gunicorn-registry stdout | 2022-05-05 10:10:13,291 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/uploads/3d53d726-da22-45f9-8594-6669c10e28e1?comp=blocklist HTTP/1.1" 201 0 gunicorn-registry stdout | 2022-05-05 10:10:13,291 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 201 gunicorn-registry stdout | 2022-05-05 10:10:13,291 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:13 GMT' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F71E3654C"' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86ac1a0-401e-0047-1568-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8dac986c-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-content-crc64': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,292 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:12 GMT' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-source': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8db154c4-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,294 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:13,392 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 202 0 gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 202 gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:13 GMT' gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F71F2CC5C"' gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86ac1c4-401e-0047-3868-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8db154c4-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,393 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,394 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-id': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,394 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,394 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:12 GMT' repositorygcworker stdout | 2022-05-05 10:10:13,394 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:10:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:24.984990+00:00 (in 11.589885 seconds) repositorygcworker stdout | 2022-05-05 10:10:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:13 UTC)" (scheduled at 2022-05-05 10:10:13.394619+00:00) gunicorn-registry stdout | 2022-05-05 10:10:13,395 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1' gunicorn-registry stdout | 2022-05-05 10:10:13,395 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'HEAD' gunicorn-registry stdout | 2022-05-05 10:10:13,395 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:13,395 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,395 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,395 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8dc0be3c-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,395 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:13,395 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' repositorygcworker stdout | 2022-05-05 10:10:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:10:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:13 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:10:13,396 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:13,417 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "HEAD /quayblobstorage/registry/sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 200 0 gunicorn-registry stdout | 2022-05-05 10:10:13,418 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 200 gunicorn-registry stdout | 2022-05-05 10:10:13,418 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:13,418 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '585146' gunicorn-registry stdout | 2022-05-05 10:10:13,418 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/octet-stream' gunicorn-registry stdout | 2022-05-05 10:10:13,418 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:13 GMT' gunicorn-registry stdout | 2022-05-05 10:10:13,418 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Accept-Ranges': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,418 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F71F2CC5C"' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86ac229-401e-0047-1b68-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8dc0be3c-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-creation-time': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-lease-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-lease-state': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-blob-type': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-id': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-source': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-progress': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-completion-time': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-access-tier': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-access-tier-inferred': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,419 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:12 GMT' gunicorn-registry stdout | 2022-05-05 10:10:13,420 [224] [DEBUG] [storage.azurestorage] Deleting chunked upload 3d53d726-da22-45f9-8594-6669c10e28e1 at path registry/uploads/3d53d726-da22-45f9-8594-6669c10e28e1 gunicorn-registry stdout | 2022-05-05 10:10:13,421 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/3d53d726-da22-45f9-8594-6669c10e28e1' gunicorn-registry stdout | 2022-05-05 10:10:13,421 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'DELETE' gunicorn-registry stdout | 2022-05-05 10:10:13,422 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:13,422 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,422 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,422 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8dc4c5b8-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,422 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:13,422 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,422 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:13,446 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "DELETE /quayblobstorage/registry/uploads/3d53d726-da22-45f9-8594-6669c10e28e1 HTTP/1.1" 202 0 gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 202 gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86ac244-401e-0047-3468-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8dc4c5b8-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-delete-type-permanent': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:13,447 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:12 GMT' gunicorn-registry stdout | 2022-05-05 10:10:13,448 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['3d53d726-da22-45f9-8594-6669c10e28e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:13,451 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', ['sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:13,453 [224] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', ['e34dca86-358b-42ee-9d63-a19067f19c94', 585146, 1532416, True, True, 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1']) gunicorn-registry stdout | 2022-05-05 10:10:13,454 [224] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [10, 9]) gunicorn-registry stdout | 2022-05-05 10:10:13,456 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:13,457 [224] [DEBUG] [peewee] ('INSERT INTO "uploadedblob" ("repository_id", "blob_id", "uploaded_at", "expires_at") VALUES (%s, %s, %s, %s) RETURNING "uploadedblob"."id"', [2, 10, datetime.datetime(2022, 5, 5, 10, 10, 13, 457596), datetime.datetime(2022, 5, 5, 11, 10, 13, 457577)]) gunicorn-registry stdout | 2022-05-05 10:10:13,459 [224] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [9]) gunicorn-registry stdout | 2022-05-05 10:10:13,465 [224] [DEBUG] [app] Ending request: urn:request:119f89fd-eed9-4b60-9c85-a79bbccf4815 (/v2/ibazulic/test-big-layers/blobs/uploads/3d53d726-da22-45f9-8594-6669c10e28e1) gunicorn-registry stdout | 2022-05-05 10:10:13,465 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:13,465 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:13,465 [224] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:13 +0000] "PUT /v2/ibazulic/test-big-layers/blobs/uploads/3d53d726-da22-45f9-8594-6669c10e28e1?digest=sha256%3A93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 201 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:13 +0000] "PUT /v2/ibazulic/test-big-layers/blobs/uploads/3d53d726-da22-45f9-8594-6669c10e28e1?digest=sha256%3A93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 201 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.247 1765 0.248) gunicorn-registry stdout | 2022-05-05 10:10:13,846 [224] [DEBUG] [app] Starting request: urn:request:fd0d9e65-3b8e-41ea-b25b-b90b9e7d5cd0 (/v2/ibazulic/test-big-layers/blobs/sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1) gunicorn-registry stdout | 2022-05-05 10:10:13,846 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:13,846 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:13,847 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:13,848 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:13,848 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:13,848 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:13,848 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:13,848 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:13,849 [224] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2 gunicorn-registry stdout | 2022-05-05 10:10:13,849 [224] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:13,851 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:13,855 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:13,857 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['e34dca86-358b-42ee-9d63-a19067f19c94']) gunicorn-registry stdout | 2022-05-05 10:10:13,860 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['e34dca86-358b-42ee-9d63-a19067f19c94']) gunicorn-registry stdout | 2022-05-05 10:10:13,862 [224] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2: {'uuid': 'e34dca86-358b-42ee-9d63-a19067f19c94', 'digest': 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 'compressed_size': 585146, 'uncompressed_size': 1532416, 'uploading': True, 'db_id': 10, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1'}} gunicorn-registry stdout | 2022-05-05 10:10:13,863 [224] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2 with expiration {'uuid': 'e34dca86-358b-42ee-9d63-a19067f19c94', 'digest': 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 'compressed_size': 585146, 'uncompressed_size': 1532416, 'uploading': True, 'db_id': 10, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1'}}: 60s gunicorn-registry stdout | 2022-05-05 10:10:13,863 [224] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__ibazulic_test-big-layers_sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1_2 with expiration {'uuid': 'e34dca86-358b-42ee-9d63-a19067f19c94', 'digest': 'sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1', 'compressed_size': 585146, 'uncompressed_size': 1532416, 'uploading': True, 'db_id': 10, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/93/93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1'}}: 60s gunicorn-registry stdout | 2022-05-05 10:10:13,864 [224] [DEBUG] [app] Ending request: urn:request:fd0d9e65-3b8e-41ea-b25b-b90b9e7d5cd0 (/v2/ibazulic/test-big-layers/blobs/sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1) gunicorn-registry stdout | 2022-05-05 10:10:13,864 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:13,864 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:13 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 200 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.019 1670 0.020) gunicorn-registry stdout | 2022-05-05 10:10:13,864 [224] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:13 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:93110824cee542506134474bc74c26ddfea83d29642668de123fca185d806dd1 HTTP/1.1" 200 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" namespacegcworker stdout | 2022-05-05 10:10:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:10:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:38.977973+00:00 (in 24.691006 seconds) namespacegcworker stdout | 2022-05-05 10:10:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:14 UTC)" (scheduled at 2022-05-05 10:10:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:10:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:10:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:10:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:10:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:44.479974+00:00 (in 29.999427 seconds) buildlogsarchiver stdout | 2022-05-05 10:10:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:44 UTC)" (scheduled at 2022-05-05 10:10:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:10:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 10, 14, 481134), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:10:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:10:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:10:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:44 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6?blockid=REDACTED&comp=REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/octet-stream' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '13736270' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8eb8476a-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-MD5': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,044 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] A body is sent with the request gunicorn-registry stdout | 2022-05-05 10:10:15,374 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6?blockid=MDAwMDA%3D&comp=block HTTP/1.1" 201 0 gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 201 gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-MD5': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86ac836-401e-0047-5c68-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8eb8476a-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,375 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:14 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,377 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:15,380 [224] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [2, 'bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 13736270, 'gASVsgAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZSMBnNoYTI1NpSMB21kX2RhdGGUQ3gfGBjJuWwHJ/uPPlr66jOyEx8mzqbU0A6sryvHTJT5G3DKjAYAAAAA/xUAAP//5CFJRgAcDwEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA4AAAAgAAAAAAAAAAAAAACUdWIu', 9, '{"blocks": ["00000"], "content-type": "application/octet-stream"}', 1, 17767424, datetime.datetime(2022, 5, 5, 10, 10, 12, 151416), None, None, 10]) gunicorn-registry stdout | 2022-05-05 10:10:15,384 [224] [DEBUG] [app] Ending request: urn:request:aef6f690-d95f-4352-8675-8013ab147bfc (/v2/ibazulic/test-big-layers/blobs/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6) gunicorn-registry stdout | 2022-05-05 10:10:15,384 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:15,384 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:15,385 [224] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:15 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6 HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:15 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6 HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (3.063 13747004 3.063) gunicorn-registry stdout | 2022-05-05 10:10:15,726 [224] [DEBUG] [app] Starting request: urn:request:3205c187-8b01-4924-b018-eed6a6d6d458 (/v2/ibazulic/test-big-layers/blobs/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6) gunicorn-registry stdout | 2022-05-05 10:10:15,726 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': 'bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:15,726 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:15,726 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:15,728 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:15,728 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:15,728 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:15,728 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:15,728 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:15,730 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:15,734 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:15,736 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:15,736 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:15,737 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:15,739 [224] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [2, 'bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 13736270, 'gASVsgAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZSMBnNoYTI1NpSMB21kX2RhdGGUQ3gfGBjJuWwHJ/uPPlr66jOyEx8mzqbU0A6sryvHTJT5G3DKjAYAAAAA/xUAAP//5CFJRgAcDwEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA4AAAAgAAAAAAAAAAAAAACUdWIu', 9, '{"blocks": ["00000"], "content-type": "application/octet-stream"}', 2, 17767424, datetime.datetime(2022, 5, 5, 10, 10, 12, 151416), None, None, 10]) gunicorn-registry stdout | 2022-05-05 10:10:15,743 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:15,743 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:15,744 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1' gunicorn-registry stdout | 2022-05-05 10:10:15,745 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'HEAD' gunicorn-registry stdout | 2022-05-05 10:10:15,745 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:15,745 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,745 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,745 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f273c60-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,745 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:15,745 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,745 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:15,772 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "HEAD /quayblobstorage/registry/sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 404 0 gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 404 gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Transfer-Encoding': 'chunked' gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86aca5f-401e-0047-6668-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f273c60-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-error-code': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,773 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:15 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,775 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6?comp=REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,775 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:15,775 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:15,775 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/xml; charset=utf-8' gunicorn-registry stdout | 2022-05-05 10:10:15,775 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,775 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-blob-content-type': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,775 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '87' gunicorn-registry stdout | 2022-05-05 10:10:15,776 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,776 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f2bed1e-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,776 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:15,776 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,776 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] A body is sent with the request gunicorn-registry stdout | 2022-05-05 10:10:15,802 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6?comp=blocklist HTTP/1.1" 201 0 gunicorn-registry stdout | 2022-05-05 10:10:15,802 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 201 gunicorn-registry stdout | 2022-05-05 10:10:15,802 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:15,802 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:15,802 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:15 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,802 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F7362A0F9"' gunicorn-registry stdout | 2022-05-05 10:10:15,802 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:15,803 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86aca75-401e-0047-7b68-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:15,803 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f2bed1e-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,803 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,803 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-content-crc64': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,803 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,803 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:15 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1' gunicorn-registry stdout | 2022-05-05 10:10:15,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:15,805 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:15,805 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-source': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,805 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,805 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,805 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f3064fc-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,805 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:15,805 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,805 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:15,840 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 202 0 gunicorn-registry stdout | 2022-05-05 10:10:15,841 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 202 gunicorn-registry stdout | 2022-05-05 10:10:15,841 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:15,841 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:15,841 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:15 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,841 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F7368457C"' gunicorn-registry stdout | 2022-05-05 10:10:15,841 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:15,841 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86aca8f-401e-0047-1368-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:15,841 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f3064fc-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,842 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,842 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-id': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,842 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,842 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:15 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1' gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'HEAD' gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f364692-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,843 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:15,864 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "HEAD /quayblobstorage/registry/sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 200 0 gunicorn-registry stdout | 2022-05-05 10:10:15,864 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 200 gunicorn-registry stdout | 2022-05-05 10:10:15,864 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:15,864 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '13736270' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/octet-stream' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:15 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Accept-Ranges': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F7368457C"' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86acaa9-401e-0047-2d68-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f364692-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-creation-time': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-lease-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-lease-state': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-blob-type': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-id': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-source': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-progress': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,865 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-completion-time': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,866 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,866 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-access-tier': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,866 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-access-tier-inferred': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,866 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:15 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,867 [224] [DEBUG] [storage.azurestorage] Deleting chunked upload bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6 at path registry/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6 gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6' gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'DELETE' gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f3a06d8-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,868 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:15,891 [224] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "DELETE /quayblobstorage/registry/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6 HTTP/1.1" 202 0 gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 202 gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'c86acab5-401e-0047-3968-606681000000' gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '8f3a06d8-cc5b-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-delete-type-permanent': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:15,892 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:15 GMT' gunicorn-registry stdout | 2022-05-05 10:10:15,893 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:15,895 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', ['sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:15,897 [224] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', ['9d63c34f-5078-4f82-bedd-61700ac05651', 13736270, 17767424, True, True, 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1']) gunicorn-registry stdout | 2022-05-05 10:10:15,898 [224] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [11, 9]) gunicorn-registry stdout | 2022-05-05 10:10:15,899 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:15,900 [224] [DEBUG] [peewee] ('INSERT INTO "uploadedblob" ("repository_id", "blob_id", "uploaded_at", "expires_at") VALUES (%s, %s, %s, %s) RETURNING "uploadedblob"."id"', [2, 11, datetime.datetime(2022, 5, 5, 10, 10, 15, 900584), datetime.datetime(2022, 5, 5, 11, 10, 15, 900573)]) gunicorn-registry stdout | 2022-05-05 10:10:15,901 [224] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [10]) gunicorn-registry stdout | 2022-05-05 10:10:15,904 [224] [DEBUG] [app] Ending request: urn:request:3205c187-8b01-4924-b018-eed6a6d6d458 (/v2/ibazulic/test-big-layers/blobs/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6) gunicorn-registry stdout | 2022-05-05 10:10:15,905 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:15,905 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. nginx stdout | 10.130.2.11 () - - [05/May/2022:10:10:15 +0000] "PUT /v2/ibazulic/test-big-layers/blobs/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6?digest=sha256%3A8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 201 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.180 1765 0.180) gunicorn-registry stdout | 2022-05-05 10:10:15,905 [224] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:10:15 +0000] "PUT /v2/ibazulic/test-big-layers/blobs/uploads/bccc5ca0-dcd0-444a-a8cd-3c6d8fd7f3f6?digest=sha256%3A8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 201 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" gunicorn-registry stdout | 2022-05-05 10:10:16,317 [224] [DEBUG] [app] Starting request: urn:request:2f188cf5-96c8-4f70-bd85-3c7381c674c3 (/v2/ibazulic/test-big-layers/blobs/sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1) gunicorn-registry stdout | 2022-05-05 10:10:16,317 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:16,317 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:16,317 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:16,319 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:16,319 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:16,319 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:16,319 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:16,319 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:16,319 [224] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2 gunicorn-registry stdout | 2022-05-05 10:10:16,319 [224] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:16,321 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:16,324 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:16,327 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['9d63c34f-5078-4f82-bedd-61700ac05651']) gunicorn-registry stdout | 2022-05-05 10:10:16,329 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['9d63c34f-5078-4f82-bedd-61700ac05651']) gunicorn-registry stdout | 2022-05-05 10:10:16,330 [224] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2: {'uuid': '9d63c34f-5078-4f82-bedd-61700ac05651', 'digest': 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 'compressed_size': 13736270, 'uncompressed_size': 17767424, 'uploading': True, 'db_id': 11, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1'}} gunicorn-registry stdout | 2022-05-05 10:10:16,331 [224] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2 with expiration {'uuid': '9d63c34f-5078-4f82-bedd-61700ac05651', 'digest': 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 'compressed_size': 13736270, 'uncompressed_size': 17767424, 'uploading': True, 'db_id': 11, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1'}}: 60s gunicorn-registry stdout | 2022-05-05 10:10:16,331 [224] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__ibazulic_test-big-layers_sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1_2 with expiration {'uuid': '9d63c34f-5078-4f82-bedd-61700ac05651', 'digest': 'sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1', 'compressed_size': 13736270, 'uncompressed_size': 17767424, 'uploading': True, 'db_id': 11, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/86/8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1'}}: 60s gunicorn-registry stdout | 2022-05-05 10:10:16,331 [224] [DEBUG] [app] Ending request: urn:request:2f188cf5-96c8-4f70-bd85-3c7381c674c3 (/v2/ibazulic/test-big-layers/blobs/sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1) gunicorn-registry stdout | 2022-05-05 10:10:16,332 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:16,332 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:16,332 [224] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:16 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 200 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:16 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:8672471369bd52072d12facd78bc30668b7c048c6cc1c2fcef4ce38031d286e1 HTTP/1.1" 200 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.016 1670 0.016) gunicorn-web stdout | 2022-05-05 10:10:17,370 [219] [DEBUG] [app] Starting request: urn:request:245f5ac1-0137-477d-95a5-792f2aab104b (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:17,370 [222] [DEBUG] [app] Starting request: urn:request:636ec004-f012-48ce-bb72-065e4d6b4420 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:17,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:10:17,373 [224] [DEBUG] [app] Starting request: urn:request:9c83a813-2df2-4998-add7-18d24248b8d6 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:17,373 [229] [DEBUG] [app] Starting request: urn:request:97f7af70-02f1-4818-bcfe-c4b8a656a14b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:17,374 [224] [DEBUG] [app] Ending request: urn:request:9c83a813-2df2-4998-add7-18d24248b8d6 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:17,374 [229] [DEBUG] [app] Ending request: urn:request:97f7af70-02f1-4818-bcfe-c4b8a656a14b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:17,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:10:17,374 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:17,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:17,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-web stdout | 2022-05-05 10:10:17,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:17,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:17,378 [219] [DEBUG] [app] Starting request: urn:request:c62fc1ee-067b-4513-a9d5-a4e746a78368 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:17,378 [221] [DEBUG] [app] Starting request: urn:request:1cfe3178-68a9-4fd7-b123-7a7ac98d1015 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:17,379 [219] [DEBUG] [app] Ending request: urn:request:c62fc1ee-067b-4513-a9d5-a4e746a78368 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:17,379 [221] [DEBUG] [app] Ending request: urn:request:1cfe3178-68a9-4fd7-b123-7a7ac98d1015 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:17,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:10:17,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:17,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:17,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:17,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:10:17,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:10:17,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:10:17,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:10:17,394 [222] [DEBUG] [app] Ending request: urn:request:636ec004-f012-48ce-bb72-065e4d6b4420 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:17,394 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:10:17,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:10:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:10:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:10:17,395 [219] [DEBUG] [app] Ending request: urn:request:245f5ac1-0137-477d-95a5-792f2aab104b (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:17,395 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:10:17,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:10:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:10:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.027) gunicorn-registry stdout | 2022-05-05 10:10:20,861 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:10:20,872 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:10:20,887 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:10:20,904 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:10:20,909 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:10:20,913 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:10:20,917 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:10:20,923 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:10:20,927 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:10:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:10:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:32.207092+00:00 (in 9.116615 seconds) notificationworker stdout | 2022-05-05 10:10:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:33 UTC)" (scheduled at 2022-05-05 10:10:23.089985+00:00) notificationworker stdout | 2022-05-05 10:10:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:10:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 23, 91016), True, datetime.datetime(2022, 5, 5, 10, 10, 23, 91016), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:10:23,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:10:23,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:10:23,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:10:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:10:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:13.394619+00:00 (in 48.409084 seconds) repositorygcworker stdout | 2022-05-05 10:10:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:24 UTC)" (scheduled at 2022-05-05 10:10:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:10:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:10:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 24, 986026), True, datetime.datetime(2022, 5, 5, 10, 10, 24, 986026), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:10:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:10:24,997 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:10:24,997 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:10:25,415 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:10:26,012 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:10:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:10:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:56.865082+00:00 (in 29.999520 seconds) gcworker stdout | 2022-05-05 10:10:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:56 UTC)" (scheduled at 2022-05-05 10:10:26.865082+00:00) gcworker stdout | 2022-05-05 10:10:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535826866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:10:26,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:10:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:10:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:10:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:10:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:58.201650+00:00 (in 29.999531 seconds) securityworker stdout | 2022-05-05 10:10:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:58 UTC)" (scheduled at 2022-05-05 10:10:28.201650+00:00) securityworker stdout | 2022-05-05 10:10:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:10:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:10:28,205 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:10:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:10:28,219 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:10:28,220 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:10:28,223 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:10:28,223 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:10:28,224 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 5, 28, 223620), 1, 2]) securityworker stdout | 2022-05-05 10:10:28,229 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:10:28,229 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:10:28,230 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 5, 28, 229972), 1, 2]) securityworker stdout | 2022-05-05 10:10:28,232 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:10:28,232 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:10:28,232 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:10:28,233 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:10:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:10:29,812 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:10:29,948 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:10:30,289 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:10:30,831 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:10:31,198 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:10:31,554 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:10:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:10:31,775 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:35.969483+00:00 (in 4.194680 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:10:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:31 UTC)" (scheduled at 2022-05-05 10:10:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:10:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:10:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 31, 775735), True, datetime.datetime(2022, 5, 5, 10, 10, 31, 775735), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:10:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:10:31,787 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:10:31,787 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:31 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:10:32,207 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:10:32,207 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:33.089985+00:00 (in 0.882374 seconds) notificationworker stdout | 2022-05-05 10:10:32,207 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:15:32 UTC)" (scheduled at 2022-05-05 10:10:32.207092+00:00) notificationworker stdout | 2022-05-05 10:10:32,208 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 10, 32, 208067), 'notification/%']) notificationworker stdout | 2022-05-05 10:10:32,219 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 10, 32, 208067), True, datetime.datetime(2022, 5, 5, 10, 10, 32, 208067), 0, 'notification/%']) notificationworker stdout | 2022-05-05 10:10:32,223 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 10, 32, 208067), True, datetime.datetime(2022, 5, 5, 10, 10, 32, 208067), 0, 'notification/%', False, datetime.datetime(2022, 5, 5, 10, 10, 32, 208067), 'notification/%']) notificationworker stdout | 2022-05-05 10:10:32,226 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:10:32,226 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:15:32 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:10:32,369 [220] [DEBUG] [app] Starting request: urn:request:3b2c2710-d487-4858-8559-9d0c2251da5d (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:32,369 [221] [DEBUG] [app] Starting request: urn:request:dfd28a51-f146-4d52-9777-478fdb092f78 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:32,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:32,371 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:10:32,372 [224] [DEBUG] [app] Starting request: urn:request:8404594c-74bc-44b3-956f-67570d23563e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:32,373 [231] [DEBUG] [app] Starting request: urn:request:2316b62a-3478-4167-a405-23897b1b324c (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:32,373 [224] [DEBUG] [app] Ending request: urn:request:8404594c-74bc-44b3-956f-67570d23563e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:32,373 [231] [DEBUG] [app] Ending request: urn:request:2316b62a-3478-4167-a405-23897b1b324c (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:32,373 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:10:32,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:32,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:32,374 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:32,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:32,376 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:10:32,377 [220] [DEBUG] [app] Starting request: urn:request:1889d3c5-ecaf-4e67-93b2-0dd1ec9c6007 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:32,377 [221] [DEBUG] [app] Starting request: urn:request:68caef07-9007-4d8b-b99d-5c16b4b5ad4b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:32,378 [220] [DEBUG] [app] Ending request: urn:request:1889d3c5-ecaf-4e67-93b2-0dd1ec9c6007 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:32,378 [221] [DEBUG] [app] Ending request: urn:request:68caef07-9007-4d8b-b99d-5c16b4b5ad4b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:32,378 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:32,378 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:10:32,379 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:32,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:32,379 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:10:32,379 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:10:32,386 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:10:32,386 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:10:32,394 [221] [DEBUG] [app] Ending request: urn:request:dfd28a51-f146-4d52-9777-478fdb092f78 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:32,395 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:10:32,395 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:10:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:10:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-web stdout | 2022-05-05 10:10:32,395 [220] [DEBUG] [app] Ending request: urn:request:3b2c2710-d487-4858-8559-9d0c2251da5d (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:32,396 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:10:32,396 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:10:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:10:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) manifestbackfillworker stdout | 2022-05-05 10:10:32,904 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:10:32,964 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} gunicorn-secscan stdout | 2022-05-05 10:10:32,969 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:10:32,974 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:10:32,978 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:10:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:10:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:43.089985+00:00 (in 9.999507 seconds) notificationworker stdout | 2022-05-05 10:10:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:43 UTC)" (scheduled at 2022-05-05 10:10:33.089985+00:00) notificationworker stdout | 2022-05-05 10:10:33,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:10:33,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 33, 90965), True, datetime.datetime(2022, 5, 5, 10, 10, 33, 90965), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:10:33,100 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:10:33,100 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:10:33,100 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:43 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:10:33,422 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:10:33,541 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:10:33,682 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:10:34,389 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:10:35,398 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:10:35,638 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:10:35,659 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:10:35,858 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:10:35,970 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:10:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:31.774027+00:00 (in 55.803634 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:10:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:35 UTC)" (scheduled at 2022-05-05 10:10:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:10:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:10:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:35 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:10:35,986 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:10:35,994 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:10:36,000 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:10:36,004 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:10:36,010 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:10:36,188 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:10:37,064 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:10:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:10:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:14.286471+00:00 (in 35.308008 seconds) namespacegcworker stdout | 2022-05-05 10:10:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:38 UTC)" (scheduled at 2022-05-05 10:10:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:10:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:10:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 38, 979015), True, datetime.datetime(2022, 5, 5, 10, 10, 38, 979015), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:10:38,989 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:10:38,989 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:10:38,989 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:10:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:10:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:53.089985+00:00 (in 9.999519 seconds) notificationworker stdout | 2022-05-05 10:10:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:53 UTC)" (scheduled at 2022-05-05 10:10:43.089985+00:00) notificationworker stdout | 2022-05-05 10:10:43,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:10:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 43, 91020), True, datetime.datetime(2022, 5, 5, 10, 10, 43, 91020), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:10:43,105 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:10:43,105 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:10:43,105 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:10:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:10:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:10:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:14.479974+00:00 (in 29.999428 seconds) buildlogsarchiver stdout | 2022-05-05 10:10:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:14 UTC)" (scheduled at 2022-05-05 10:10:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:10:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 10, 44, 481067), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:10:44,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:10:44,490 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:10:44,490 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:10:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:10:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:10:59.184038+00:00 (in 11.996011 seconds) exportactionlogsworker stdout | 2022-05-05 10:10:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:47 UTC)" (scheduled at 2022-05-05 10:10:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:10:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:10:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:11:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:10:47,369 [222] [DEBUG] [app] Starting request: urn:request:321a083e-0e86-4f70-907a-38d0ce2e45d1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:47,369 [219] [DEBUG] [app] Starting request: urn:request:530abea3-56f3-4bb5-8e06-9e2c3fc01706 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:47,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:10:47,373 [231] [DEBUG] [app] Starting request: urn:request:7c1e4f4e-bf14-4603-be41-45ccbd55c44f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:47,373 [224] [DEBUG] [app] Starting request: urn:request:279e7368-9891-4b89-bbb2-dac0507cf8f2 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:47,374 [224] [DEBUG] [app] Ending request: urn:request:279e7368-9891-4b89-bbb2-dac0507cf8f2 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:47,374 [231] [DEBUG] [app] Ending request: urn:request:7c1e4f4e-bf14-4603-be41-45ccbd55c44f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:10:47,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:10:47,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:47,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:47,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.003) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-web stdout | 2022-05-05 10:10:47,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:47,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:10:47,378 [219] [DEBUG] [app] Starting request: urn:request:5aa41e24-c3f4-4d95-a62d-0b41084f759b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:47,379 [219] [DEBUG] [app] Ending request: urn:request:5aa41e24-c3f4-4d95-a62d-0b41084f759b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:47,379 [221] [DEBUG] [app] Starting request: urn:request:32822af4-9d82-4d14-bd0b-25d770c9a9ed (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.001) gunicorn-web stdout | 2022-05-05 10:10:47,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:47,380 [221] [DEBUG] [app] Ending request: urn:request:32822af4-9d82-4d14-bd0b-25d770c9a9ed (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:10:47,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:10:47,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:10:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:10:47,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:10:47,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:10:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:10:47,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:10:47,386 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:10:47,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:10:47,394 [222] [DEBUG] [app] Ending request: urn:request:321a083e-0e86-4f70-907a-38d0ce2e45d1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:47,394 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:10:47,395 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:10:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:10:47,395 [219] [DEBUG] [app] Ending request: urn:request:530abea3-56f3-4bb5-8e06-9e2c3fc01706 (/health/instance) gunicorn-web stdout | 2022-05-05 10:10:47,395 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:10:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:10:47,395 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:10:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:10:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.027) gunicorn-registry stdout | 2022-05-05 10:10:50,872 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:10:50,882 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:10:50,898 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:10:50,913 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:10:50,922 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:10:50,927 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:10:50,930 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:10:50,935 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:10:50,939 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:10:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:10:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:03.089985+00:00 (in 9.999510 seconds) notificationworker stdout | 2022-05-05 10:10:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:03 UTC)" (scheduled at 2022-05-05 10:10:53.089985+00:00) notificationworker stdout | 2022-05-05 10:10:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:10:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 10, 53, 90910), True, datetime.datetime(2022, 5, 5, 10, 10, 53, 90910), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:10:53,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:10:53,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:10:53,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:10:55,444 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} gunicorn-registry stdout | 2022-05-05 10:10:55,868 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035?blockid=REDACTED&comp=REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:55,868 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:55,868 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/octet-stream' gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '54941777' gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a70112de-cc5b-11ec-a969-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-MD5': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:55,869 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] A body is sent with the request gunicorn-registry stdout | 2022-05-05 10:10:55,872 [231] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaystorageaccount2.blob.core.windows.net:443 expiredappspecifictokenworker stdout | 2022-05-05 10:10:56,043 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:10:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:10:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:26.865082+00:00 (in 29.999356 seconds) gcworker stdout | 2022-05-05 10:10:56,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:26 UTC)" (scheduled at 2022-05-05 10:10:56.865082+00:00) gcworker stdout | 2022-05-05 10:10:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535856866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:10:56,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:10:56,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:10:56,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:26 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:10:57,634 [231] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035?blockid=MDAwMDA%3D&comp=block HTTP/1.1" 201 0 gunicorn-registry stdout | 2022-05-05 10:10:57,634 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 201 gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-MD5': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': 'ec7a5713-c01e-003b-7068-60487e000000' gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a70112de-cc5b-11ec-a969-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:57,635 [231] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:57 GMT' gunicorn-registry stdout | 2022-05-05 10:10:57,637 [231] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:57,640 [231] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [2, 'ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 54941777, 'gASVsgAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZSMBnNoYTI1NpSMB21kX2RhdGGUQ3jV0+BkhkyNqT11khTTzS7acKN6LOI4tUCgdUIzA0PzWojCMhoAAAAAOZny/wIAAP//3N4xrAAgsgcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABEAAAAgAAAAAAAAAAAAAACUdWIu', 9, '{"blocks": ["00000"], "content-type": "application/octet-stream"}', 1, 129114112, datetime.datetime(2022, 5, 5, 10, 10, 12, 192211), None, None, 13]) gunicorn-registry stdout | 2022-05-05 10:10:57,645 [231] [DEBUG] [app] Ending request: urn:request:959c7215-195d-4228-b25d-c5b0de290e0b (/v2/ibazulic/test-big-layers/blobs/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035) gunicorn-registry stdout | 2022-05-05 10:10:57,645 [231] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:57,645 [231] [DEBUG] [peewee.pool] Returning 140384019515536 to pool. gunicorn-registry stdout | 2022-05-05 10:10:57,646 [231] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:57 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035 HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:57 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035 HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (45.190 54972187 45.188) gunicorn-registry stdout | 2022-05-05 10:10:58,105 [229] [DEBUG] [app] Starting request: urn:request:cb7fd165-9761-472b-ab01-1c6e077abc06 (/v2/ibazulic/test-big-layers/blobs/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035) gunicorn-registry stdout | 2022-05-05 10:10:58,105 [229] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': 'ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:58,105 [229] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:58,105 [229] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:58,106 [229] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:58,106 [229] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:58,107 [229] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:58,107 [229] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:58,107 [229] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:58,108 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:58,112 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:58,116 [229] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:58,116 [229] [DEBUG] [peewee.pool] Returning 140384010638288 to pool. gunicorn-registry stdout | 2022-05-05 10:10:58,117 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:58,121 [229] [DEBUG] [peewee] ('UPDATE "blobupload" SET "repository_id" = %s, "uuid" = %s, "byte_count" = %s, "sha_state" = %s, "location_id" = %s, "storage_metadata" = %s, "chunk_count" = %s, "uncompressed_byte_count" = %s, "created" = %s, "piece_sha_state" = %s, "piece_hashes" = %s WHERE ("blobupload"."id" = %s)', [2, 'ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 54941777, 'gASVsgAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZSMBnNoYTI1NpSMB21kX2RhdGGUQ3jV0+BkhkyNqT11khTTzS7acKN6LOI4tUCgdUIzA0PzWojCMhoAAAAAOZny/wIAAP//3N4xrAAgsgcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABEAAAAgAAAAAAAAAAAAAACUdWIu', 9, '{"blocks": ["00000"], "content-type": "application/octet-stream"}', 2, 129114112, datetime.datetime(2022, 5, 5, 10, 10, 12, 192211), None, None, 13]) gunicorn-registry stdout | 2022-05-05 10:10:58,124 [229] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:58,124 [229] [DEBUG] [peewee.pool] Returning 140384010638288 to pool. gunicorn-registry stdout | 2022-05-05 10:10:58,126 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a' gunicorn-registry stdout | 2022-05-05 10:10:58,126 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'HEAD' gunicorn-registry stdout | 2022-05-05 10:10:58,126 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:58,127 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,127 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,127 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a86a2e44-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,127 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:58,127 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,127 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:58,130 [229] [DEBUG] [urllib3.connectionpool] Starting new HTTPS connection (1): quaystorageaccount2.blob.core.windows.net:443 securityworker stdout | 2022-05-05 10:10:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:10:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:28.201650+00:00 (in 29.999489 seconds) securityworker stdout | 2022-05-05 10:10:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:28 UTC)" (scheduled at 2022-05-05 10:10:58.201650+00:00) securityworker stdout | 2022-05-05 10:10:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:10:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:10:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:10:58,208 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:10:58,218 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:10:58,218 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:28 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:10:58,225 [229] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "HEAD /quayblobstorage/registry/sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 404 0 gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 404 gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Transfer-Encoding': 'chunked' gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': '82f6ea80-801e-0067-5368-601d26000000' gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a86a2e44-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-error-code': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,226 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:58 GMT' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035?comp=REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/xml; charset=utf-8' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-blob-content-type': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '87' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a879b62a-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,228 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] A body is sent with the request gunicorn-registry stdout | 2022-05-05 10:10:58,253 [229] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035?comp=blocklist HTTP/1.1" 201 0 gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 201 gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:58 GMT' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F8CB061DD"' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': '82f6ea93-801e-0067-6368-601d26000000' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a879b62a-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-content-crc64': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,254 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:58 GMT' gunicorn-registry stdout | 2022-05-05 10:10:58,256 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a' gunicorn-registry stdout | 2022-05-05 10:10:58,256 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:10:58,256 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:58,256 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-source': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,256 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,256 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,256 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a87e0734-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,256 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:58,257 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,257 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:58,313 [229] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "PUT /quayblobstorage/registry/sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 202 0 gunicorn-registry stdout | 2022-05-05 10:10:58,313 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 202 gunicorn-registry stdout | 2022-05-05 10:10:58,313 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:58,313 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:58,313 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:58 GMT' gunicorn-registry stdout | 2022-05-05 10:10:58,314 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F8CB9132B"' gunicorn-registry stdout | 2022-05-05 10:10:58,314 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:58,314 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': '82f6eaa6-801e-0067-7368-601d26000000' gunicorn-registry stdout | 2022-05-05 10:10:58,314 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a87e0734-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,314 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,314 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-id': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,314 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,314 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:58 GMT' gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a' gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'HEAD' gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a8870384-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,315 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:58,336 [229] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "HEAD /quayblobstorage/registry/sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 200 0 gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 200 gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '54941777' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/octet-stream' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Last-Modified': 'Thu, 05 May 2022 10:10:58 GMT' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Accept-Ranges': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'ETag': '"0x8DA2E7F8CB9132B"' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': '82f6eace-801e-0067-1868-601d26000000' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a8870384-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-creation-time': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-lease-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-lease-state': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-blob-type': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-id': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-source': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-status': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-progress': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,337 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-copy-completion-time': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,338 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-server-encrypted': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,338 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-access-tier': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,338 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-access-tier-inferred': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,338 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:58 GMT' gunicorn-registry stdout | 2022-05-05 10:10:58,339 [229] [DEBUG] [storage.azurestorage] Deleting chunked upload ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035 at path registry/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035 gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035' gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'DELETE' gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a88abeb6-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,340 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] No body was attached to the request gunicorn-registry stdout | 2022-05-05 10:10:58,364 [229] [DEBUG] [urllib3.connectionpool] https://quaystorageaccount2.blob.core.windows.net:443 "DELETE /quayblobstorage/registry/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035 HTTP/1.1" 202 0 gunicorn-registry stdout | 2022-05-05 10:10:58,364 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response status: 202 gunicorn-registry stdout | 2022-05-05 10:10:58,364 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] Response headers: gunicorn-registry stdout | 2022-05-05 10:10:58,364 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '0' gunicorn-registry stdout | 2022-05-05 10:10:58,364 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0' gunicorn-registry stdout | 2022-05-05 10:10:58,365 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-request-id': '82f6eadc-801e-0067-2468-601d26000000' gunicorn-registry stdout | 2022-05-05 10:10:58,365 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': 'a88abeb6-cc5b-11ec-b639-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:10:58,365 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,365 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-delete-type-permanent': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:10:58,365 [229] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Date': 'Thu, 05 May 2022 10:10:58 GMT' gunicorn-registry stdout | 2022-05-05 10:10:58,366 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:58,369 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."image_size", "t1"."uncompressed_size", "t1"."uploading", "t1"."cas_path", "t1"."content_checksum" FROM "imagestorage" AS "t1" WHERE ("t1"."content_checksum" = %s) LIMIT %s OFFSET %s', ['sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:58,371 [229] [DEBUG] [peewee] ('INSERT INTO "imagestorage" ("uuid", "image_size", "uncompressed_size", "uploading", "cas_path", "content_checksum") VALUES (%s, %s, %s, %s, %s, %s) RETURNING "imagestorage"."id"', ['9af5e9bb-a306-4462-a174-7cfa78d9fb4b', 54941777, 129114112, True, True, 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a']) gunicorn-registry stdout | 2022-05-05 10:10:58,372 [229] [DEBUG] [peewee] ('INSERT INTO "imagestorageplacement" ("storage_id", "location_id") VALUES (%s, %s) RETURNING "imagestorageplacement"."id"', [12, 9]) gunicorn-registry stdout | 2022-05-05 10:10:58,374 [229] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:58,375 [229] [DEBUG] [peewee] ('INSERT INTO "uploadedblob" ("repository_id", "blob_id", "uploaded_at", "expires_at") VALUES (%s, %s, %s, %s) RETURNING "uploadedblob"."id"', [2, 12, datetime.datetime(2022, 5, 5, 10, 10, 58, 375527), datetime.datetime(2022, 5, 5, 11, 10, 58, 375508)]) gunicorn-registry stdout | 2022-05-05 10:10:58,377 [229] [DEBUG] [peewee] ('DELETE FROM "blobupload" WHERE ("blobupload"."id" = %s)', [13]) gunicorn-registry stdout | 2022-05-05 10:10:58,380 [229] [DEBUG] [app] Ending request: urn:request:cb7fd165-9761-472b-ab01-1c6e077abc06 (/v2/ibazulic/test-big-layers/blobs/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035) gunicorn-registry stdout | 2022-05-05 10:10:58,381 [229] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:58,381 [229] [DEBUG] [peewee.pool] Returning 140384010638288 to pool. gunicorn-registry stdout | 2022-05-05 10:10:58,381 [229] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:58 +0000] "PUT /v2/ibazulic/test-big-layers/blobs/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035?digest=sha256%3A6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 201 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:58 +0000] "PUT /v2/ibazulic/test-big-layers/blobs/uploads/ba05eac3-6e10-41e0-9c0d-ea2eb1dd0035?digest=sha256%3A6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 201 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.277 1765 0.278) gunicorn-registry stdout | 2022-05-05 10:10:58,808 [224] [DEBUG] [app] Starting request: urn:request:5390e10c-4a07-40bb-86b3-c47a62c1eb1a (/v2/ibazulic/test-big-layers/blobs/sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a) gunicorn-registry stdout | 2022-05-05 10:10:58,808 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'digest': 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:10:58,808 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:58,808 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NDExLCJpYXQiOjE2NTE3NDU0MTEsImV4cCI6MTY1MTc0OTAxMSwic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.J28xV___QZI1RP5pCQLVVhzLX6Zs92-RuG2tj3FVAvFwpQND25R224r1_x_6a740c4H2SfUqjsw3AuYtkmpbenw6C6CRZramDswwK4T-RoXOIs3Uvwa5KWDDCOpCgIOkj89tlNQrqo4FKZyDJotgsieNFenTrcIHoa0Ma9B02UZMbN1DdTjdZqhkp-CS6IRYZF37XYxub7g44IGNiAB-If5hmCQ4ru82s-GZCSob4cNYsgJCDaEQLkSXztds46GTPCs2mX7KLKV24YRqpkWTDYd3jLcejki1ONAnrrBOW_JyMOnbQ_rLJL_-KTB-0P16HCG4Il44kXjWx2jC_fZKZg gunicorn-registry stdout | 2022-05-05 10:10:58,810 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:10:58,810 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:10:58,810 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:10:58,810 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:10:58,810 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:10:58,810 [224] [DEBUG] [data.cache.impl] Checking cache for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2 gunicorn-registry stdout | 2022-05-05 10:10:58,810 [224] [DEBUG] [data.cache.impl] Found no result in cache for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2; calling loader gunicorn-registry stdout | 2022-05-05 10:10:58,811 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:58,814 [224] [DEBUG] [peewee] ('SELECT "t1"."uuid" FROM "imagestorage" AS "t1" INNER JOIN "uploadedblob" AS "t2" ON ("t2"."blob_id" = "t1"."id") WHERE (("t2"."repository_id" = %s) AND ("t1"."content_checksum" = %s)) LIMIT %s OFFSET %s', [2, 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:10:58,817 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id", "t2"."id", "t2"."uuid", "t2"."image_size", "t2"."uncompressed_size", "t2"."uploading", "t2"."cas_path", "t2"."content_checksum" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['9af5e9bb-a306-4462-a174-7cfa78d9fb4b']) gunicorn-registry stdout | 2022-05-05 10:10:58,819 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."storage_id", "t1"."location_id" FROM "imagestorageplacement" AS "t1" INNER JOIN "imagestorage" AS "t2" ON ("t1"."storage_id" = "t2"."id") WHERE ("t2"."uuid" = %s)', ['9af5e9bb-a306-4462-a174-7cfa78d9fb4b']) gunicorn-registry stdout | 2022-05-05 10:10:58,821 [224] [DEBUG] [data.cache.impl] Got loaded result for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2: {'uuid': '9af5e9bb-a306-4462-a174-7cfa78d9fb4b', 'digest': 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 'compressed_size': 54941777, 'uncompressed_size': 129114112, 'uploading': True, 'db_id': 12, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a'}} gunicorn-registry stdout | 2022-05-05 10:10:58,821 [224] [DEBUG] [data.cache.impl] Caching loaded result for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2 with expiration {'uuid': '9af5e9bb-a306-4462-a174-7cfa78d9fb4b', 'digest': 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 'compressed_size': 54941777, 'uncompressed_size': 129114112, 'uploading': True, 'db_id': 12, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a'}}: 60s gunicorn-registry stdout | 2022-05-05 10:10:58,821 [224] [DEBUG] [data.cache.impl] Cached loaded result for key repo_blob__ibazulic_test-big-layers_sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a_2 with expiration {'uuid': '9af5e9bb-a306-4462-a174-7cfa78d9fb4b', 'digest': 'sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a', 'compressed_size': 54941777, 'uncompressed_size': 129114112, 'uploading': True, 'db_id': 12, 'inputs': {'placements': ['azure'], 'storage_path': 'sha256/6a/6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a'}}: 60s gunicorn-registry stdout | 2022-05-05 10:10:58,822 [224] [DEBUG] [app] Ending request: urn:request:5390e10c-4a07-40bb-86b3-c47a62c1eb1a (/v2/ibazulic/test-big-layers/blobs/sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a) gunicorn-registry stdout | 2022-05-05 10:10:58,822 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:10:58,822 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:10:58,822 [224] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:10:58 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 200 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:10:58 +0000] "HEAD /v2/ibazulic/test-big-layers/blobs/sha256:6aefca2dc61dcbcd268b8a9861e552f9cdb69e57242faec64ac120d2355a9c1a HTTP/1.1" 200 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.015 1670 0.016) exportactionlogsworker stdout | 2022-05-05 10:10:59,184 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:10:59,184 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:00.978909+00:00 (in 1.794365 seconds) exportactionlogsworker stdout | 2022-05-05 10:10:59,184 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:15:59 UTC)" (scheduled at 2022-05-05 10:10:59.184038+00:00) exportactionlogsworker stdout | 2022-05-05 10:10:59,185 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 10, 59, 185068), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:10:59,195 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 10, 59, 185068), True, datetime.datetime(2022, 5, 5, 10, 10, 59, 185068), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:10:59,198 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 10, 59, 185068), True, datetime.datetime(2022, 5, 5, 10, 10, 59, 185068), 0, 'exportactionlogs/%', False, datetime.datetime(2022, 5, 5, 10, 10, 59, 185068), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:10:59,201 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:10:59,201 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:15:59 UTC)" executed successfully gcworker stdout | 2022-05-05 10:10:59,842 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:10:59,975 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:11:00,327 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:11:00,859 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:11:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:11:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:47.187465+00:00 (in 46.208027 seconds) exportactionlogsworker stdout | 2022-05-05 10:11:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:00 UTC)" (scheduled at 2022-05-05 10:11:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:11:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:11:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 0, 979973), True, datetime.datetime(2022, 5, 5, 10, 11, 0, 979973), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:11:00,992 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:11:00,992 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:11:00,992 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:11:01,235 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:11:01,591 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:11:02,369 [220] [DEBUG] [app] Starting request: urn:request:d42c2042-29a5-4123-8423-13a4ab5ad3cc (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:02,370 [219] [DEBUG] [app] Starting request: urn:request:92fbc2ca-93c8-44bb-824d-edd9f1566d8f (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:02,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:02,371 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:11:02,373 [230] [DEBUG] [app] Starting request: urn:request:8db798d2-62b3-4733-81b4-e7460a44f25e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:02,373 [231] [DEBUG] [app] Starting request: urn:request:e7c2c9e2-0638-4a47-bb1d-458e8d7f988a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:02,374 [231] [DEBUG] [app] Ending request: urn:request:e7c2c9e2-0638-4a47-bb1d-458e8d7f988a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:02,374 [230] [DEBUG] [app] Ending request: urn:request:8db798d2-62b3-4733-81b4-e7460a44f25e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:02,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:11:02,375 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:11:02,375 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:02,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:11:02,377 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:02,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:02,378 [219] [DEBUG] [app] Starting request: urn:request:9dc1ca44-082c-4b5f-9e24-332bb5f813d2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:02,379 [219] [DEBUG] [app] Ending request: urn:request:9dc1ca44-082c-4b5f-9e24-332bb5f813d2 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:11:02,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:11:02,379 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:02,380 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:11:02,381 [222] [DEBUG] [app] Starting request: urn:request:adef048a-3252-45d0-bb83-63d39ef4f8eb (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:02,381 [222] [DEBUG] [app] Ending request: urn:request:adef048a-3252-45d0-bb83-63d39ef4f8eb (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:02,382 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:11:02,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:02,383 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:11:02,386 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:11:02,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:11:02,394 [220] [DEBUG] [app] Ending request: urn:request:d42c2042-29a5-4123-8423-13a4ab5ad3cc (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:02,394 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:11:02,395 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:11:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:11:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:11:02,396 [219] [DEBUG] [app] Ending request: urn:request:92fbc2ca-93c8-44bb-824d-edd9f1566d8f (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:02,396 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:11:02,397 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:11:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:11:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) manifestbackfillworker stdout | 2022-05-05 10:11:02,941 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} gunicorn-secscan stdout | 2022-05-05 10:11:02,977 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:11:02,983 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:11:02,986 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} builder stdout | 2022-05-05 10:11:03,000 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} notificationworker stdout | 2022-05-05 10:11:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:11:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:12.207990+00:00 (in 9.117349 seconds) notificationworker stdout | 2022-05-05 10:11:03,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:13 UTC)" (scheduled at 2022-05-05 10:11:03.089985+00:00) notificationworker stdout | 2022-05-05 10:11:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:11:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 3, 91207), True, datetime.datetime(2022, 5, 5, 10, 11, 3, 91207), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:11:03,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:11:03,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:11:03,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:13 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:11:03,459 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:11:03,575 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:11:03,693 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} blobuploadcleanupworker stdout | 2022-05-05 10:11:04,069 [55] [DEBUG] [apscheduler.scheduler] Looking for jobs to run blobuploadcleanupworker stdout | 2022-05-05 10:11:04,069 [55] [INFO] [apscheduler.executors.default] Running job "BlobUploadCleanupWorker._try_cleanup_uploads (trigger: interval[1:00:00], next run at: 2022-05-05 10:11:04 UTC)" (scheduled at 2022-05-05 10:11:04.068940+00:00) blobuploadcleanupworker stdout | 2022-05-05 10:11:04,069 [55] [DEBUG] [util.locking] Acquiring global lock BLOB_CLEANUP blobuploadcleanupworker stdout | 2022-05-05 10:11:04,070 [55] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 11:11:04.068940+00:00 (in 3599.999607 seconds) blobuploadcleanupworker stdout | 2022-05-05 10:11:04,070 [55] [DEBUG] [util.locking] Acquired lock BLOB_CLEANUP blobuploadcleanupworker stdout | 2022-05-05 10:11:04,070 [55] [DEBUG] [__main__] Performing blob upload cleanup blobuploadcleanupworker stdout | 2022-05-05 10:11:04,071 [55] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."created" <= %s) LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 3, 10, 11, 4, 70807), 1, 0]) blobuploadcleanupworker stdout | 2022-05-05 10:11:04,082 [55] [DEBUG] [__main__] No additional stale blob uploads found blobuploadcleanupworker stdout | 2022-05-05 10:11:04,082 [55] [DEBUG] [data.database] Disconnecting from database. blobuploadcleanupworker stdout | 2022-05-05 10:11:04,083 [55] [DEBUG] [util.locking] Releasing lock BLOB_CLEANUP blobuploadcleanupworker stdout | 2022-05-05 10:11:04,083 [55] [DEBUG] [util.locking] Released lock BLOB_CLEANUP blobuploadcleanupworker stdout | 2022-05-05 10:11:04,083 [55] [INFO] [apscheduler.executors.default] Job "BlobUploadCleanupWorker._try_cleanup_uploads (trigger: interval[1:00:00], next run at: 2022-05-05 11:11:04 UTC)" executed successfully repositoryactioncounter stdout | 2022-05-05 10:11:04,417 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:11:05,435 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:11:05,655 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:11:05,674 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:11:05,894 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:11:05,995 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:11:06,002 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:11:06,008 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:11:06,012 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:11:06,017 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:11:06,211 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:11:07,076 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:11:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:11:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:13.089985+00:00 (in 0.881416 seconds) notificationworker stdout | 2022-05-05 10:11:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:12 UTC)" (scheduled at 2022-05-05 10:11:12.207990+00:00) notificationworker stdout | 2022-05-05 10:11:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:11:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:11:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:11:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:23.089985+00:00 (in 9.999503 seconds) notificationworker stdout | 2022-05-05 10:11:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:23 UTC)" (scheduled at 2022-05-05 10:11:13.089985+00:00) notificationworker stdout | 2022-05-05 10:11:13,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:11:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 13, 91064), True, datetime.datetime(2022, 5, 5, 10, 11, 13, 91064), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:11:13,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:11:13,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:11:13,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:11:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:11:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:24.984990+00:00 (in 11.589878 seconds) repositorygcworker stdout | 2022-05-05 10:11:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:13 UTC)" (scheduled at 2022-05-05 10:11:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:11:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:11:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:11:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:11:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:38.977973+00:00 (in 24.691010 seconds) namespacegcworker stdout | 2022-05-05 10:11:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:14 UTC)" (scheduled at 2022-05-05 10:11:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:11:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:11:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:11:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:11:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:44.479974+00:00 (in 29.999423 seconds) buildlogsarchiver stdout | 2022-05-05 10:11:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:44 UTC)" (scheduled at 2022-05-05 10:11:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:11:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 11, 14, 480956), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:11:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:11:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:11:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:11:17,370 [222] [DEBUG] [app] Starting request: urn:request:b15cfbe7-7005-40a1-9908-69f792076770 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:17,370 [219] [DEBUG] [app] Starting request: urn:request:38e0a103-f4d0-43ee-b746-3a04dfcb90a6 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:17,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:11:17,373 [230] [DEBUG] [app] Starting request: urn:request:793570cf-cb87-4da6-b313-a65cf12439b6 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:17,374 [224] [DEBUG] [app] Starting request: urn:request:d898bdce-2775-478c-a81e-6ce6602f4fd3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:17,374 [230] [DEBUG] [app] Ending request: urn:request:793570cf-cb87-4da6-b313-a65cf12439b6 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:17,375 [224] [DEBUG] [app] Ending request: urn:request:d898bdce-2775-478c-a81e-6ce6602f4fd3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:17,375 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:11:17,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-web stdout | 2022-05-05 10:11:17,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:17,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:17,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:17,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:17,379 [220] [DEBUG] [app] Starting request: urn:request:10f7b95e-6b8e-4e46-8e1a-d57ed9d8d7fc (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:17,379 [221] [DEBUG] [app] Starting request: urn:request:484dcbd5-4645-4b78-a327-385835f46080 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:17,379 [220] [DEBUG] [app] Ending request: urn:request:10f7b95e-6b8e-4e46-8e1a-d57ed9d8d7fc (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:17,380 [221] [DEBUG] [app] Ending request: urn:request:484dcbd5-4645-4b78-a327-385835f46080 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:17,380 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:11:17,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:11:17,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:17,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:11:17,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:11:17,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:11:17,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:11:17,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:11:17,393 [219] [DEBUG] [app] Ending request: urn:request:38e0a103-f4d0-43ee-b746-3a04dfcb90a6 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:17,393 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:11:17,394 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:11:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:11:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:11:17,395 [222] [DEBUG] [app] Ending request: urn:request:b15cfbe7-7005-40a1-9908-69f792076770 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:17,395 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:11:17,395 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:11:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:11:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-registry stdout | 2022-05-05 10:11:20,884 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:11:20,891 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:11:20,909 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:11:20,924 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:11:20,934 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:11:20,938 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:11:20,942 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:11:20,946 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:11:20,951 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:11:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:11:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:33.089985+00:00 (in 9.999455 seconds) notificationworker stdout | 2022-05-05 10:11:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:33 UTC)" (scheduled at 2022-05-05 10:11:23.089985+00:00) notificationworker stdout | 2022-05-05 10:11:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:11:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 23, 91012), True, datetime.datetime(2022, 5, 5, 10, 11, 23, 91012), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:11:23,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:11:23,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:11:23,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:11:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:11:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:13.394619+00:00 (in 48.409165 seconds) repositorygcworker stdout | 2022-05-05 10:11:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:24 UTC)" (scheduled at 2022-05-05 10:11:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:11:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:11:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 24, 985992), True, datetime.datetime(2022, 5, 5, 10, 11, 24, 985992), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:11:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:11:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:11:24,997 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:11:25,482 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:11:26,064 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:11:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:11:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:56.865082+00:00 (in 29.999503 seconds) gcworker stdout | 2022-05-05 10:11:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:56 UTC)" (scheduled at 2022-05-05 10:11:26.865082+00:00) gcworker stdout | 2022-05-05 10:11:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535886865, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:11:26,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:11:26,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:11:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:11:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:11:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:58.201650+00:00 (in 29.999458 seconds) securityworker stdout | 2022-05-05 10:11:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:58 UTC)" (scheduled at 2022-05-05 10:11:28.201650+00:00) securityworker stdout | 2022-05-05 10:11:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:11:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:11:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:11:28,208 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:11:28,217 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:11:28,220 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:11:28,223 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:11:28,223 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:11:28,223 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 6, 28, 223259), 1, 2]) securityworker stdout | 2022-05-05 10:11:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:11:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:11:28,226 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 6, 28, 225845), 1, 2]) securityworker stdout | 2022-05-05 10:11:28,228 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:11:28,228 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:11:28,228 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:11:28,228 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:11:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:11:29,880 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:11:30,015 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:11:30,346 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:11:30,892 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:11:31,272 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:11:31,617 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:11:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:11:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:35.969483+00:00 (in 4.194913 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:11:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:31 UTC)" (scheduled at 2022-05-05 10:11:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:11:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:11:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 31, 775109), True, datetime.datetime(2022, 5, 5, 10, 11, 31, 775109), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:11:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:11:31,786 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:11:31,786 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:11:32,369 [220] [DEBUG] [app] Starting request: urn:request:f53bc29c-2a17-408a-b86d-382a6ba09423 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:32,369 [221] [DEBUG] [app] Starting request: urn:request:b883d769-c77c-4dc0-9b6a-ca7cdb203175 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:32,372 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:32,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:11:32,373 [224] [DEBUG] [app] Starting request: urn:request:e225932a-908d-43bd-bae2-44b172585b2e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:32,374 [231] [DEBUG] [app] Starting request: urn:request:e16e68a2-80b1-416b-8cc6-19ef0f571e7e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:32,374 [224] [DEBUG] [app] Ending request: urn:request:e225932a-908d-43bd-bae2-44b172585b2e (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:11:32,374 [231] [DEBUG] [app] Ending request: urn:request:e16e68a2-80b1-416b-8cc6-19ef0f571e7e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:32,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:11:32,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:11:32,375 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:11:32,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:32,377 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:32,378 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:32,378 [220] [DEBUG] [app] Starting request: urn:request:63a2c2ec-086c-471d-abb4-b54596a7a1a6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:32,379 [220] [DEBUG] [app] Ending request: urn:request:63a2c2ec-086c-471d-abb4-b54596a7a1a6 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:11:32,379 [219] [DEBUG] [app] Starting request: urn:request:9ddc3636-0437-4143-98ad-d0433be35af7 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:32,380 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:11:32,380 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:32,380 [219] [DEBUG] [app] Ending request: urn:request:9ddc3636-0437-4143-98ad-d0433be35af7 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:32,380 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:11:32,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:11:32,381 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:32,381 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:11:32,387 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:11:32,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:11:32,393 [220] [DEBUG] [app] Ending request: urn:request:f53bc29c-2a17-408a-b86d-382a6ba09423 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:32,394 [220] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:11:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.025) gunicorn-web stdout | 2022-05-05 10:11:32,394 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:11:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:11:32,394 [221] [DEBUG] [app] Ending request: urn:request:b883d769-c77c-4dc0-9b6a-ca7cdb203175 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:32,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:11:32,395 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:11:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:11:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.026) manifestbackfillworker stdout | 2022-05-05 10:11:32,950 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} gunicorn-secscan stdout | 2022-05-05 10:11:32,986 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:11:32,993 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:11:32,997 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} builder stdout | 2022-05-05 10:11:33,037 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} notificationworker stdout | 2022-05-05 10:11:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:11:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:43.089985+00:00 (in 9.999461 seconds) notificationworker stdout | 2022-05-05 10:11:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:43 UTC)" (scheduled at 2022-05-05 10:11:33.089985+00:00) notificationworker stdout | 2022-05-05 10:11:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:11:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 33, 91079), True, datetime.datetime(2022, 5, 5, 10, 11, 33, 91079), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:11:33,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:11:33,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:11:33,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:43 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:11:33,473 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:11:33,611 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:11:33,728 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:11:34,454 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:11:35,473 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:11:35,672 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:11:35,711 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:11:35,931 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:11:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:11:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:31.774027+00:00 (in 55.803975 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:11:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:35 UTC)" (scheduled at 2022-05-05 10:11:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:11:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:11:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:35 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:11:36,004 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:11:36,012 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:11:36,020 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:11:36,026 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:11:36,030 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:11:36,234 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:11:37,112 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:11:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:11:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:14.286471+00:00 (in 35.307780 seconds) namespacegcworker stdout | 2022-05-05 10:11:38,979 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:38 UTC)" (scheduled at 2022-05-05 10:11:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:11:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:11:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 38, 979109), True, datetime.datetime(2022, 5, 5, 10, 11, 38, 979109), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:11:38,991 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:11:38,991 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:11:38,991 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:11:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:11:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:11:53.089985+00:00 (in 9.999440 seconds) notificationworker stdout | 2022-05-05 10:11:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:53 UTC)" (scheduled at 2022-05-05 10:11:43.089985+00:00) notificationworker stdout | 2022-05-05 10:11:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:11:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 43, 91107), True, datetime.datetime(2022, 5, 5, 10, 11, 43, 91107), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:11:43,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:11:43,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:11:43,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:11:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:11:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:11:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:14.479974+00:00 (in 29.999503 seconds) buildlogsarchiver stdout | 2022-05-05 10:11:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:14 UTC)" (scheduled at 2022-05-05 10:11:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:11:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 11, 44, 480952), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:11:44,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:11:44,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:11:44,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:11:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:11:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:00.978909+00:00 (in 13.790980 seconds) exportactionlogsworker stdout | 2022-05-05 10:11:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:47 UTC)" (scheduled at 2022-05-05 10:11:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:11:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:11:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:12:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:11:47,370 [219] [DEBUG] [app] Starting request: urn:request:0462a7f3-4b0c-4151-af8a-1c4dac85d2d7 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:47,370 [222] [DEBUG] [app] Starting request: urn:request:5a58e3ed-a800-453f-a05b-b2ebfd3a8fad (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:47,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:47,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:11:47,374 [229] [DEBUG] [app] Starting request: urn:request:eb50b8b5-9113-49c4-b0a9-ee4d83ebd5ee (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:47,374 [231] [DEBUG] [app] Starting request: urn:request:be347fbd-8a3c-468b-9230-d219c9e51c60 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:47,374 [231] [DEBUG] [app] Ending request: urn:request:be347fbd-8a3c-468b-9230-d219c9e51c60 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:47,374 [229] [DEBUG] [app] Ending request: urn:request:eb50b8b5-9113-49c4-b0a9-ee4d83ebd5ee (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:11:47,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:11:47,375 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:11:47,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:47,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:47,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:47,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:11:47,378 [222] [DEBUG] [app] Starting request: urn:request:426d1619-29a1-4f61-9b7c-59b296ea254f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:47,379 [222] [DEBUG] [app] Ending request: urn:request:426d1619-29a1-4f61-9b7c-59b296ea254f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:47,379 [221] [DEBUG] [app] Starting request: urn:request:fafd738b-3e42-440d-a9d2-a78bc50c70ba (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:11:47,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:11:47,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:11:47,380 [221] [DEBUG] [app] Ending request: urn:request:fafd738b-3e42-440d-a9d2-a78bc50c70ba (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:11:47,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:11:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:11:47,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:11:47,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:11:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:11:47,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:11:47,389 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:11:47,390 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:11:47,396 [219] [DEBUG] [app] Ending request: urn:request:0462a7f3-4b0c-4151-af8a-1c4dac85d2d7 (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:47,396 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:11:47,396 [222] [DEBUG] [app] Ending request: urn:request:5a58e3ed-a800-453f-a05b-b2ebfd3a8fad (/health/instance) gunicorn-web stdout | 2022-05-05 10:11:47,396 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:11:47,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:11:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:11:47,397 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:11:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:11:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:11:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.028) gunicorn-registry stdout | 2022-05-05 10:11:50,895 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:11:50,900 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:11:50,920 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:11:50,935 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:11:50,946 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:11:50,949 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:11:50,954 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:11:50,959 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:11:50,962 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:11:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:11:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:03.089985+00:00 (in 9.999399 seconds) notificationworker stdout | 2022-05-05 10:11:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:03 UTC)" (scheduled at 2022-05-05 10:11:53.089985+00:00) notificationworker stdout | 2022-05-05 10:11:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:11:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 11, 53, 91103), True, datetime.datetime(2022, 5, 5, 10, 11, 53, 91103), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:11:53,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:11:53,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:11:53,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:11:55,503 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:11:56,102 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:11:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:11:56,866 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:26.865082+00:00 (in 29.999304 seconds) gcworker stdout | 2022-05-05 10:11:56,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:26 UTC)" (scheduled at 2022-05-05 10:11:56.865082+00:00) gcworker stdout | 2022-05-05 10:11:56,867 [62] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100]) gcworker stdout | 2022-05-05 10:11:56,876 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535916875, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:11:56,880 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:11:56,880 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:11:56,880 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:11:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:11:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:28.201650+00:00 (in 29.999524 seconds) securityworker stdout | 2022-05-05 10:11:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:28 UTC)" (scheduled at 2022-05-05 10:11:58.201650+00:00) securityworker stdout | 2022-05-05 10:11:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:11:58,203 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:11:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:11:58,208 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:11:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:11:58,216 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:11:59,896 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:12:00,025 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:12:00,382 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:12:00,929 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:12:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:12:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:47.187465+00:00 (in 46.207999 seconds) exportactionlogsworker stdout | 2022-05-05 10:12:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:00 UTC)" (scheduled at 2022-05-05 10:12:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:12:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:12:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 0, 979953), True, datetime.datetime(2022, 5, 5, 10, 12, 0, 979953), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:12:00,991 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:12:00,991 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:12:00,991 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:12:01,310 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:12:01,654 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:12:02,370 [219] [DEBUG] [app] Starting request: urn:request:52cbe1d8-b619-4680-a719-4dd04e120ba3 (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:02,370 [220] [DEBUG] [app] Starting request: urn:request:89ace9f8-fc1e-484b-bcd0-96e0bcbebe3b (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:02,372 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:02,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:12:02,374 [224] [DEBUG] [app] Starting request: urn:request:6c6ec09a-a622-481e-a53e-f78ecb746094 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:02,374 [228] [DEBUG] [app] Starting request: urn:request:e96f320e-5bd7-41a2-8d80-e2d656abe812 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:02,374 [224] [DEBUG] [app] Ending request: urn:request:6c6ec09a-a622-481e-a53e-f78ecb746094 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:12:02,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:12:02,375 [228] [DEBUG] [app] Ending request: urn:request:e96f320e-5bd7-41a2-8d80-e2d656abe812 (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:02,375 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:12:02,375 [228] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:02,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-web stdout | 2022-05-05 10:12:02,377 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:02,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:02,378 [219] [DEBUG] [app] Starting request: urn:request:39aa800e-5296-49b4-abee-f1a6094b493e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:02,379 [219] [DEBUG] [app] Ending request: urn:request:39aa800e-5296-49b4-abee-f1a6094b493e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:02,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:12:02,379 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:02,380 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:12:02,381 [222] [DEBUG] [app] Starting request: urn:request:906df06e-80e0-490b-8c2e-e316e6bfcc2e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:02,382 [222] [DEBUG] [app] Ending request: urn:request:906df06e-80e0-490b-8c2e-e316e6bfcc2e (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:12:02,382 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:02,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:02,383 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:12:02,387 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:12:02,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:12:02,394 [220] [DEBUG] [app] Ending request: urn:request:89ace9f8-fc1e-484b-bcd0-96e0bcbebe3b (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:02,395 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:12:02,395 [219] [DEBUG] [app] Ending request: urn:request:52cbe1d8-b619-4680-a719-4dd04e120ba3 (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:02,395 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:12:02,395 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:12:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:12:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:12:02,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:12:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:12:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.027) manifestbackfillworker stdout | 2022-05-05 10:12:02,982 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} gunicorn-secscan stdout | 2022-05-05 10:12:02,995 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:12:03,004 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:12:03,007 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} builder stdout | 2022-05-05 10:12:03,074 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} notificationworker stdout | 2022-05-05 10:12:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:12:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:12.207990+00:00 (in 9.117506 seconds) notificationworker stdout | 2022-05-05 10:12:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:13 UTC)" (scheduled at 2022-05-05 10:12:03.089985+00:00) notificationworker stdout | 2022-05-05 10:12:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:12:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 3, 90976), True, datetime.datetime(2022, 5, 5, 10, 12, 3, 90976), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:12:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:12:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:12:03,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:13 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:12:03,511 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:12:03,621 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:12:03,757 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:12:04,494 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:12:05,510 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:12:05,705 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:12:05,744 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:12:05,963 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:12:06,012 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:12:06,022 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:12:06,028 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:12:06,034 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:12:06,037 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:12:06,261 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:12:07,149 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:12:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:12:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:13.089985+00:00 (in 0.881490 seconds) notificationworker stdout | 2022-05-05 10:12:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:12 UTC)" (scheduled at 2022-05-05 10:12:12.207990+00:00) notificationworker stdout | 2022-05-05 10:12:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:12:12,208 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:12:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:12:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:23.089985+00:00 (in 9.999504 seconds) notificationworker stdout | 2022-05-05 10:12:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:23 UTC)" (scheduled at 2022-05-05 10:12:13.089985+00:00) notificationworker stdout | 2022-05-05 10:12:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:12:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 13, 90953), True, datetime.datetime(2022, 5, 5, 10, 12, 13, 90953), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:12:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:12:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:12:13,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:12:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:12:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:24.984990+00:00 (in 11.589825 seconds) repositorygcworker stdout | 2022-05-05 10:12:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:13 UTC)" (scheduled at 2022-05-05 10:12:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:12:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:12:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:12:14,287 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:12:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:38.977973+00:00 (in 24.690769 seconds) namespacegcworker stdout | 2022-05-05 10:12:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:14 UTC)" (scheduled at 2022-05-05 10:12:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:12:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:12:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:12:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:12:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:44.479974+00:00 (in 29.999510 seconds) buildlogsarchiver stdout | 2022-05-05 10:12:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:44 UTC)" (scheduled at 2022-05-05 10:12:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:12:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 12, 14, 481020), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:12:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:12:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:12:14,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:12:17,369 [221] [DEBUG] [app] Starting request: urn:request:2d4446d6-98c4-4e97-bc31-283c93b26152 (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:17,369 [222] [DEBUG] [app] Starting request: urn:request:7628b23f-ad42-4a13-bbba-66187a9a458d (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:17,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:12:17,372 [224] [DEBUG] [app] Starting request: urn:request:59bae52d-c8cd-4284-ab1a-e5f1c7fc1c65 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:17,373 [225] [DEBUG] [app] Starting request: urn:request:104796be-5804-4134-bd7f-61e7a3a5bc9a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:17,373 [224] [DEBUG] [app] Ending request: urn:request:59bae52d-c8cd-4284-ab1a-e5f1c7fc1c65 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:17,373 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:12:17,374 [225] [DEBUG] [app] Ending request: urn:request:104796be-5804-4134-bd7f-61e7a3a5bc9a (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:17,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:12:17,374 [225] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:17,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:17,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:17,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:17,378 [222] [DEBUG] [app] Starting request: urn:request:0802e44d-0799-437e-aab0-781b3037d2c2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:17,378 [219] [DEBUG] [app] Starting request: urn:request:97ea316b-8ce5-4f3d-b862-c652d4827c9f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:17,378 [222] [DEBUG] [app] Ending request: urn:request:0802e44d-0799-437e-aab0-781b3037d2c2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:17,378 [219] [DEBUG] [app] Ending request: urn:request:97ea316b-8ce5-4f3d-b862-c652d4827c9f (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:12:17,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:17,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:17,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:17,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:12:17,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:12:17,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:12:17,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:12:17,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:12:17,395 [221] [DEBUG] [app] Ending request: urn:request:2d4446d6-98c4-4e97-bc31-283c93b26152 (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:17,395 [222] [DEBUG] [app] Ending request: urn:request:7628b23f-ad42-4a13-bbba-66187a9a458d (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:17,395 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:12:17,395 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:12:17,396 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:12:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:12:17,396 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:12:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:12:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.028) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:12:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-registry stdout | 2022-05-05 10:12:20,905 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:12:20,910 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:12:20,930 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:12:20,944 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:12:20,954 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:12:20,960 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:12:20,965 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:12:20,969 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:12:20,972 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:12:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:12:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:33.089985+00:00 (in 9.999523 seconds) notificationworker stdout | 2022-05-05 10:12:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:33 UTC)" (scheduled at 2022-05-05 10:12:23.089985+00:00) notificationworker stdout | 2022-05-05 10:12:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:12:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 23, 91034), True, datetime.datetime(2022, 5, 5, 10, 12, 23, 91034), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:12:23,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:12:23,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:12:23,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:12:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:12:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:00.385766+00:00 (in 35.400303 seconds) repositorygcworker stdout | 2022-05-05 10:12:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:24 UTC)" (scheduled at 2022-05-05 10:12:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:12:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:12:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 24, 985988), True, datetime.datetime(2022, 5, 5, 10, 12, 24, 985988), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:12:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:12:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:12:24,997 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:12:25,540 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:12:26,139 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:12:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:12:26,866 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:56.865082+00:00 (in 29.999225 seconds) gcworker stdout | 2022-05-05 10:12:26,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:56 UTC)" (scheduled at 2022-05-05 10:12:26.865082+00:00) gcworker stdout | 2022-05-05 10:12:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535946866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:12:26,881 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:12:26,881 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:12:26,881 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:12:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:12:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:58.201650+00:00 (in 29.999453 seconds) securityworker stdout | 2022-05-05 10:12:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:58 UTC)" (scheduled at 2022-05-05 10:12:28.201650+00:00) securityworker stdout | 2022-05-05 10:12:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:12:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:12:28,205 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:12:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:12:28,215 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:12:28,217 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:12:28,220 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:12:28,220 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:12:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 7, 28, 220518), 1, 2]) securityworker stdout | 2022-05-05 10:12:28,223 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:12:28,223 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:12:28,223 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 7, 28, 223227), 1, 2]) securityworker stdout | 2022-05-05 10:12:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:12:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:12:28,225 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:12:28,225 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:12:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:12:29,922 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:12:30,053 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:12:30,414 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:12:30,966 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:12:31,346 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:12:31,666 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:12:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:12:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:35.969483+00:00 (in 4.194908 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:12:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:31 UTC)" (scheduled at 2022-05-05 10:12:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:12:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:12:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 31, 775063), True, datetime.datetime(2022, 5, 5, 10, 12, 31, 775063), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:12:31,785 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:12:31,785 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:12:31,786 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:12:32,369 [220] [DEBUG] [app] Starting request: urn:request:db6e7ec7-51ff-4727-b987-04a3c6fa5d9d (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:32,369 [222] [DEBUG] [app] Starting request: urn:request:929087ed-ca8a-4e67-ba20-e0a300863fe5 (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:32,370 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:32,371 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:12:32,372 [224] [DEBUG] [app] Starting request: urn:request:85bcb4a6-cc9c-4847-9a45-5ef10b99842d (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:32,372 [229] [DEBUG] [app] Starting request: urn:request:af663a6a-ad6c-4732-9c37-148b56ea8f96 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:32,373 [229] [DEBUG] [app] Ending request: urn:request:af663a6a-ad6c-4732-9c37-148b56ea8f96 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:32,373 [224] [DEBUG] [app] Ending request: urn:request:85bcb4a6-cc9c-4847-9a45-5ef10b99842d (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:32,373 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:32,374 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:12:32,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:32,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:12:32,375 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:32,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:32,377 [220] [DEBUG] [app] Starting request: urn:request:e7a00ae9-a859-4f3e-af6c-d3a8dc8a556b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:32,378 [222] [DEBUG] [app] Starting request: urn:request:6f237382-1314-435f-89d1-4ee8818d6f86 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:32,378 [220] [DEBUG] [app] Ending request: urn:request:e7a00ae9-a859-4f3e-af6c-d3a8dc8a556b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:32,378 [222] [DEBUG] [app] Ending request: urn:request:6f237382-1314-435f-89d1-4ee8818d6f86 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:32,378 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:12:32,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:32,379 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:32,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:32,379 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:12:32,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:12:32,385 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:12:32,387 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:12:32,393 [222] [DEBUG] [app] Ending request: urn:request:929087ed-ca8a-4e67-ba20-e0a300863fe5 (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:32,393 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:12:32,393 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:12:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:12:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:12:32,394 [220] [DEBUG] [app] Ending request: urn:request:db6e7ec7-51ff-4727-b987-04a3c6fa5d9d (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:32,394 [220] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:12:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-web stdout | 2022-05-05 10:12:32,395 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:12:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-secscan stdout | 2022-05-05 10:12:33,004 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:12:33,011 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:12:33,015 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:12:33,019 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} notificationworker stdout | 2022-05-05 10:12:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:12:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:43.089985+00:00 (in 9.999254 seconds) notificationworker stdout | 2022-05-05 10:12:33,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:43 UTC)" (scheduled at 2022-05-05 10:12:33.089985+00:00) notificationworker stdout | 2022-05-05 10:12:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:12:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 33, 91202), True, datetime.datetime(2022, 5, 5, 10, 12, 33, 91202), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:12:33,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:12:33,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:12:33,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:43 UTC)" executed successfully builder stdout | 2022-05-05 10:12:33,111 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:12:33,541 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:12:33,658 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:12:33,797 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:12:34,530 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:12:35,546 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:12:35,727 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:12:35,781 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} securityscanningnotificationworker stdout | 2022-05-05 10:12:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:12:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:03.965762+00:00 (in 27.995778 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:12:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:35 UTC)" (scheduled at 2022-05-05 10:12:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:12:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:12:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:35 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:12:35,999 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:12:36,021 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:12:36,031 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:12:36,036 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:12:36,044 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:12:36,048 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:12:36,281 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:12:37,186 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:12:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:12:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:14.286471+00:00 (in 35.307975 seconds) namespacegcworker stdout | 2022-05-05 10:12:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:38 UTC)" (scheduled at 2022-05-05 10:12:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:12:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:12:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 38, 979068), True, datetime.datetime(2022, 5, 5, 10, 12, 38, 979068), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:12:38,989 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:12:38,989 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:12:38,989 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:12:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:12:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:12:53.089985+00:00 (in 9.999502 seconds) notificationworker stdout | 2022-05-05 10:12:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:53 UTC)" (scheduled at 2022-05-05 10:12:43.089985+00:00) notificationworker stdout | 2022-05-05 10:12:43,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:12:43,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 43, 90931), True, datetime.datetime(2022, 5, 5, 10, 12, 43, 90931), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:12:43,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:12:43,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:12:43,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:12:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:12:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:12:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:14.479974+00:00 (in 29.999435 seconds) buildlogsarchiver stdout | 2022-05-05 10:12:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:14 UTC)" (scheduled at 2022-05-05 10:12:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:12:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 12, 44, 481039), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:12:44,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:12:44,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:12:44,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:12:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:12:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:00.978909+00:00 (in 13.790928 seconds) exportactionlogsworker stdout | 2022-05-05 10:12:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:47 UTC)" (scheduled at 2022-05-05 10:12:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:12:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:12:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:13:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:12:47,369 [222] [DEBUG] [app] Starting request: urn:request:6d1805a1-0813-4bbb-b916-97216e90e207 (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:47,369 [221] [DEBUG] [app] Starting request: urn:request:bdfa9d70-99f4-4cca-b318-bfa7bc1aac3e (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:47,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:12:47,373 [224] [DEBUG] [app] Starting request: urn:request:7fff6c2e-f40c-4fad-bf54-4083a6623571 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:47,373 [229] [DEBUG] [app] Starting request: urn:request:a6fb0929-96f9-41a6-aa31-7a70403e67ec (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:47,374 [224] [DEBUG] [app] Ending request: urn:request:7fff6c2e-f40c-4fad-bf54-4083a6623571 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:12:47,374 [229] [DEBUG] [app] Ending request: urn:request:a6fb0929-96f9-41a6-aa31-7a70403e67ec (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-registry stdout | 2022-05-05 10:12:47,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:12:47,374 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:47,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:47,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:12:47,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:12:47,379 [221] [DEBUG] [app] Starting request: urn:request:973cbbde-5241-4e01-8297-d5a8cc8e9e12 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:47,379 [221] [DEBUG] [app] Ending request: urn:request:973cbbde-5241-4e01-8297-d5a8cc8e9e12 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:47,379 [222] [DEBUG] [app] Starting request: urn:request:def2ad81-956f-4708-b6c7-1aa82a600a1f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:47,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:12:47,380 [222] [DEBUG] [app] Ending request: urn:request:def2ad81-956f-4708-b6c7-1aa82a600a1f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:12:47,380 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:47,380 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:12:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:12:47,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [05/May/2022:10:12:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:12:47,381 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:12:47,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:12:47,386 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:12:47,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:12:47,393 [221] [DEBUG] [app] Ending request: urn:request:bdfa9d70-99f4-4cca-b318-bfa7bc1aac3e (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:47,393 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:12:47,393 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:12:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:12:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.024) gunicorn-web stdout | 2022-05-05 10:12:47,394 [222] [DEBUG] [app] Ending request: urn:request:6d1805a1-0813-4bbb-b916-97216e90e207 (/health/instance) gunicorn-web stdout | 2022-05-05 10:12:47,394 [222] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:12:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.026) gunicorn-web stdout | 2022-05-05 10:12:47,395 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:12:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:12:50,916 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:12:50,920 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:12:50,939 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:12:50,953 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:12:50,963 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:12:50,975 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:12:50,980 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} gunicorn-registry stdout | 2022-05-05 10:12:50,983 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:12:50,987 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} notificationworker stdout | 2022-05-05 10:12:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:12:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:03.089985+00:00 (in 9.999451 seconds) notificationworker stdout | 2022-05-05 10:12:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:03 UTC)" (scheduled at 2022-05-05 10:12:53.089985+00:00) notificationworker stdout | 2022-05-05 10:12:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:12:53,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 12, 53, 91086), True, datetime.datetime(2022, 5, 5, 10, 12, 53, 91086), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:12:53,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:12:53,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:12:53,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:12:55,561 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:12:56,157 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:12:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:12:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:26.865082+00:00 (in 29.999515 seconds) gcworker stdout | 2022-05-05 10:12:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:26 UTC)" (scheduled at 2022-05-05 10:12:56.865082+00:00) gcworker stdout | 2022-05-05 10:12:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650535976866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:12:56,879 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:12:56,879 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:12:56,879 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:12:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:12:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:28.201650+00:00 (in 29.999536 seconds) securityworker stdout | 2022-05-05 10:12:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:28 UTC)" (scheduled at 2022-05-05 10:12:58.201650+00:00) securityworker stdout | 2022-05-05 10:12:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:12:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:12:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:12:58,208 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:12:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:12:58,216 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:12:59,959 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:13:00,090 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} repositorygcworker stdout | 2022-05-05 10:13:00,386 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:13:00,386 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:13.394619+00:00 (in 13.008371 seconds) repositorygcworker stdout | 2022-05-05 10:13:00,386 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:18:00 UTC)" (scheduled at 2022-05-05 10:13:00.385766+00:00) repositorygcworker stdout | 2022-05-05 10:13:00,387 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 13, 0, 386696), 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:13:00,395 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 13, 0, 386696), True, datetime.datetime(2022, 5, 5, 10, 13, 0, 386696), 0, 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:13:00,398 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 13, 0, 386696), True, datetime.datetime(2022, 5, 5, 10, 13, 0, 386696), 0, 'repositorygc/%', False, datetime.datetime(2022, 5, 5, 10, 13, 0, 386696), 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:13:00,400 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:13:00,400 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:18:00 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:13:00,438 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:13:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:13:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:47.187465+00:00 (in 46.208073 seconds) exportactionlogsworker stdout | 2022-05-05 10:13:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:00 UTC)" (scheduled at 2022-05-05 10:13:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:13:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:13:00,980 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 0, 979878), True, datetime.datetime(2022, 5, 5, 10, 13, 0, 979878), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:13:00,989 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:13:00,989 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:13:00,989 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:00 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:13:01,004 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:13:01,370 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:13:01,704 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:13:02,370 [222] [DEBUG] [app] Starting request: urn:request:63e361b3-843a-4be1-8791-dc4a0fc1027a (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:02,370 [219] [DEBUG] [app] Starting request: urn:request:51b896fb-ba02-4fd7-8771-ef085dbc80e3 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:02,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:02,373 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:13:02,374 [224] [DEBUG] [app] Starting request: urn:request:c6c00f40-0c99-43b2-bb5e-7b023a8f485d (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:02,375 [225] [DEBUG] [app] Starting request: urn:request:837bf9fc-024c-4bb5-9730-674bd9b80c39 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:02,375 [224] [DEBUG] [app] Ending request: urn:request:c6c00f40-0c99-43b2-bb5e-7b023a8f485d (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:02,375 [225] [DEBUG] [app] Ending request: urn:request:837bf9fc-024c-4bb5-9730-674bd9b80c39 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:02,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-registry stdout | 2022-05-05 10:13:02,376 [225] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:02,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:02,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:13:02,378 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:02,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:02,379 [219] [DEBUG] [app] Starting request: urn:request:be160d10-241f-45ae-b336-3d0191b5607c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:02,380 [221] [DEBUG] [app] Starting request: urn:request:2185239d-4fb1-4c1f-9f8f-61a25b847b4e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:02,380 [219] [DEBUG] [app] Ending request: urn:request:be160d10-241f-45ae-b336-3d0191b5607c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:02,381 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:02,381 [221] [DEBUG] [app] Ending request: urn:request:2185239d-4fb1-4c1f-9f8f-61a25b847b4e (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:13:02,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:02,381 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:13:02,381 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:02,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:13:02,382 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:13:02,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:13:02,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:13:02,394 [219] [DEBUG] [app] Ending request: urn:request:51b896fb-ba02-4fd7-8771-ef085dbc80e3 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:02,394 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:13:02,395 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:13:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:13:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.026) gunicorn-web stdout | 2022-05-05 10:13:02,396 [222] [DEBUG] [app] Ending request: urn:request:63e361b3-843a-4be1-8791-dc4a0fc1027a (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:02,396 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:13:02,396 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:13:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:13:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.028) gunicorn-secscan stdout | 2022-05-05 10:13:03,012 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:13:03,019 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:13:03,024 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:13:03,052 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} notificationworker stdout | 2022-05-05 10:13:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:13:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:12.207990+00:00 (in 9.117460 seconds) notificationworker stdout | 2022-05-05 10:13:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:13 UTC)" (scheduled at 2022-05-05 10:13:03.089985+00:00) notificationworker stdout | 2022-05-05 10:13:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:13:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 3, 91036), True, datetime.datetime(2022, 5, 5, 10, 13, 3, 91036), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:13:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:13:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:13:03,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:13 UTC)" executed successfully builder stdout | 2022-05-05 10:13:03,148 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:13:03,577 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:13:03,698 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:13:03,834 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} securityscanningnotificationworker stdout | 2022-05-05 10:13:03,966 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:13:03,966 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:31.774027+00:00 (in 27.807771 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:13:03,966 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:18:03 UTC)" (scheduled at 2022-05-05 10:13:03.965762+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:13:03,967 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 13, 3, 966747), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:13:03,976 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 13, 3, 966747), True, datetime.datetime(2022, 5, 5, 10, 13, 3, 966747), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:13:03,978 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 13, 3, 966747), True, datetime.datetime(2022, 5, 5, 10, 13, 3, 966747), 0, 'secscanv4/%', False, datetime.datetime(2022, 5, 5, 10, 13, 3, 966747), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:13:03,980 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:13:03,980 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:18:03 UTC)" executed successfully repositoryactioncounter stdout | 2022-05-05 10:13:04,568 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:13:05,583 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:13:05,735 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:13:05,817 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:13:06,031 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} notificationworker stdout | 2022-05-05 10:13:06,037 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} gunicorn-web stdout | 2022-05-05 10:13:06,042 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:13:06,045 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:13:06,053 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:13:06,057 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} securityscanningnotificationworker stdout | 2022-05-05 10:13:06,319 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:13:07,219 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:13:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:13:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:13.089985+00:00 (in 0.881481 seconds) notificationworker stdout | 2022-05-05 10:13:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:12 UTC)" (scheduled at 2022-05-05 10:13:12.207990+00:00) notificationworker stdout | 2022-05-05 10:13:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:13:12,208 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:13:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:13:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:23.089985+00:00 (in 9.999504 seconds) notificationworker stdout | 2022-05-05 10:13:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:23 UTC)" (scheduled at 2022-05-05 10:13:13.089985+00:00) notificationworker stdout | 2022-05-05 10:13:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:13:13,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 13, 90900), True, datetime.datetime(2022, 5, 5, 10, 13, 13, 90900), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:13:13,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:13:13,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:13:13,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:13:13,394 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:13:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:24.984990+00:00 (in 11.589907 seconds) repositorygcworker stdout | 2022-05-05 10:13:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:13 UTC)" (scheduled at 2022-05-05 10:13:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:13:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:13:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:13:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:13:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:38.977973+00:00 (in 24.690932 seconds) namespacegcworker stdout | 2022-05-05 10:13:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:14 UTC)" (scheduled at 2022-05-05 10:13:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:13:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:13:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:13:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:13:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:44.479974+00:00 (in 29.999503 seconds) buildlogsarchiver stdout | 2022-05-05 10:13:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:44 UTC)" (scheduled at 2022-05-05 10:13:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:13:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 13, 14, 480917), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:13:14,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:13:14,490 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:13:14,490 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:13:17,369 [222] [DEBUG] [app] Starting request: urn:request:cda24ca6-c717-41e8-87e4-d77d06e80655 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:17,369 [219] [DEBUG] [app] Starting request: urn:request:5a3ce0aa-bab2-44ec-8163-40eb44d12276 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:17,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:13:17,372 [225] [DEBUG] [app] Starting request: urn:request:334fa175-50ce-41d5-b14b-830a12835954 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:17,372 [231] [DEBUG] [app] Starting request: urn:request:40d381a4-db36-4c0b-9ded-7669bf16d7fa (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:17,373 [225] [DEBUG] [app] Ending request: urn:request:334fa175-50ce-41d5-b14b-830a12835954 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:17,373 [231] [DEBUG] [app] Ending request: urn:request:40d381a4-db36-4c0b-9ded-7669bf16d7fa (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:13:17,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:17,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:17,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:17,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:17,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:13:17,376 [225] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:17,377 [222] [DEBUG] [app] Starting request: urn:request:43ed7ec7-cf8f-47c0-8ec1-98c16125de4d (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:17,377 [221] [DEBUG] [app] Starting request: urn:request:52be8abe-3485-42f1-80ec-319c691d1109 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:17,378 [222] [DEBUG] [app] Ending request: urn:request:43ed7ec7-cf8f-47c0-8ec1-98c16125de4d (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:17,378 [221] [DEBUG] [app] Ending request: urn:request:52be8abe-3485-42f1-80ec-319c691d1109 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:13:17,378 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:17,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:17,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:13:17,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:17,379 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:13:17,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:13:17,385 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:13:17,385 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:13:17,392 [219] [DEBUG] [app] Ending request: urn:request:5a3ce0aa-bab2-44ec-8163-40eb44d12276 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:17,392 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:13:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.024 118 0.025) gunicorn-web stdout | 2022-05-05 10:13:17,392 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:13:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:13:17,393 [222] [DEBUG] [app] Ending request: urn:request:cda24ca6-c717-41e8-87e4-d77d06e80655 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:17,394 [222] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:13:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:13:17,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:13:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:13:20,931 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:13:20,938 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:13:20,950 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:13:20,967 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:13:20,975 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:13:20,985 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:13:20,992 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:13:20,997 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} gunicorn-registry stdout | 2022-05-05 10:13:21,498 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} notificationworker stdout | 2022-05-05 10:13:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:13:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:33.089985+00:00 (in 9.999502 seconds) notificationworker stdout | 2022-05-05 10:13:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:33 UTC)" (scheduled at 2022-05-05 10:13:23.089985+00:00) notificationworker stdout | 2022-05-05 10:13:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:13:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 23, 90915), True, datetime.datetime(2022, 5, 5, 10, 13, 23, 90915), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:13:23,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:13:23,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:13:23,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:13:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:13:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:13.394619+00:00 (in 48.409168 seconds) repositorygcworker stdout | 2022-05-05 10:13:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:24 UTC)" (scheduled at 2022-05-05 10:13:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:13:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:13:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 24, 985977), True, datetime.datetime(2022, 5, 5, 10, 13, 24, 985977), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:13:24,997 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:13:24,998 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:13:24,998 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:13:25,591 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:13:26,176 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:13:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:13:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:56.865082+00:00 (in 29.999530 seconds) gcworker stdout | 2022-05-05 10:13:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:56 UTC)" (scheduled at 2022-05-05 10:13:26.865082+00:00) gcworker stdout | 2022-05-05 10:13:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536006866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:13:26,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:13:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:13:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:13:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:13:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:58.201650+00:00 (in 29.999517 seconds) securityworker stdout | 2022-05-05 10:13:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:58 UTC)" (scheduled at 2022-05-05 10:13:28.201650+00:00) securityworker stdout | 2022-05-05 10:13:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:13:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:13:28,205 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:13:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:13:28,217 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:13:28,219 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:13:28,222 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:13:28,222 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:13:28,222 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 8, 28, 222222), 1, 2]) securityworker stdout | 2022-05-05 10:13:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:13:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:13:28,225 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 8, 28, 225216), 1, 2]) securityworker stdout | 2022-05-05 10:13:28,228 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:13:28,228 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:13:28,228 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:13:28,228 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:13:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:13:29,996 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:13:30,127 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:13:30,475 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:13:31,042 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:13:31,406 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:13:31,730 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:13:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:13:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:35.969483+00:00 (in 4.194981 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:13:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:31 UTC)" (scheduled at 2022-05-05 10:13:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:13:31,774 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:13:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 31, 774986), True, datetime.datetime(2022, 5, 5, 10, 13, 31, 774986), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:13:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:13:31,786 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:13:31,787 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:13:32,369 [221] [DEBUG] [app] Starting request: urn:request:44c59d0e-aa7d-45a4-af16-14dd89719a48 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:32,369 [219] [DEBUG] [app] Starting request: urn:request:ba97eeda-ae16-44c1-a236-4a42390f700a (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:32,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:32,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:13:32,372 [229] [DEBUG] [app] Starting request: urn:request:d5a2153d-4625-4829-9d7c-c0d56f68ee87 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:32,373 [230] [DEBUG] [app] Starting request: urn:request:a9ec23fa-652a-4c82-b10f-d347d1ad8b41 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:32,373 [229] [DEBUG] [app] Ending request: urn:request:d5a2153d-4625-4829-9d7c-c0d56f68ee87 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:32,373 [230] [DEBUG] [app] Ending request: urn:request:a9ec23fa-652a-4c82-b10f-d347d1ad8b41 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:32,373 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:13:32,374 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:32,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:32,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:13:32,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:32,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:32,378 [221] [DEBUG] [app] Starting request: urn:request:e9cd3bf5-7492-4250-8bb0-c98f912d6205 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:32,378 [221] [DEBUG] [app] Ending request: urn:request:e9cd3bf5-7492-4250-8bb0-c98f912d6205 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:13:32,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:32,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:32,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:13:32,381 [222] [DEBUG] [app] Starting request: urn:request:be92883e-e8f9-450b-9542-349050f12bbf (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:32,381 [222] [DEBUG] [app] Ending request: urn:request:be92883e-e8f9-450b-9542-349050f12bbf (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:32,382 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:13:32,382 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:32,383 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:13:32,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:13:32,389 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:13:32,395 [221] [DEBUG] [app] Ending request: urn:request:44c59d0e-aa7d-45a4-af16-14dd89719a48 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:32,395 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:13:32,395 [219] [DEBUG] [app] Ending request: urn:request:ba97eeda-ae16-44c1-a236-4a42390f700a (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:32,396 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:13:32,396 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:13:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:13:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.028) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:13:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-web stdout | 2022-05-05 10:13:32,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:13:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-secscan stdout | 2022-05-05 10:13:33,023 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:13:33,027 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:13:33,032 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:13:33,088 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} notificationworker stdout | 2022-05-05 10:13:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:13:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:43.089985+00:00 (in 9.999496 seconds) notificationworker stdout | 2022-05-05 10:13:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:43 UTC)" (scheduled at 2022-05-05 10:13:33.089985+00:00) notificationworker stdout | 2022-05-05 10:13:33,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:13:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 33, 90976), True, datetime.datetime(2022, 5, 5, 10, 13, 33, 90976), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:13:33,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:13:33,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:13:33,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:43 UTC)" executed successfully builder stdout | 2022-05-05 10:13:33,185 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:13:33,603 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:13:33,729 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:13:33,872 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:13:34,593 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:13:35,617 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:13:35,771 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:13:35,837 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} securityscanningnotificationworker stdout | 2022-05-05 10:13:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:13:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:31.774027+00:00 (in 55.804056 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:13:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:35 UTC)" (scheduled at 2022-05-05 10:13:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:13:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:13:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:35 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:13:36,040 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:13:36,053 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:13:36,058 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:13:36,063 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:13:36,067 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:13:36,073 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:13:36,356 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:13:37,245 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:13:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:13:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:14.286471+00:00 (in 35.308024 seconds) namespacegcworker stdout | 2022-05-05 10:13:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:38 UTC)" (scheduled at 2022-05-05 10:13:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:13:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:13:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 38, 978994), True, datetime.datetime(2022, 5, 5, 10, 13, 38, 978994), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:13:38,990 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:13:38,990 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:13:38,990 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:13:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:13:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:13:53.089985+00:00 (in 9.999509 seconds) notificationworker stdout | 2022-05-05 10:13:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:53 UTC)" (scheduled at 2022-05-05 10:13:43.089985+00:00) notificationworker stdout | 2022-05-05 10:13:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:13:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 43, 91039), True, datetime.datetime(2022, 5, 5, 10, 13, 43, 91039), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:13:43,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:13:43,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:13:43,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:13:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:13:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:13:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:14.479974+00:00 (in 29.999520 seconds) buildlogsarchiver stdout | 2022-05-05 10:13:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:14 UTC)" (scheduled at 2022-05-05 10:13:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:13:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 13, 44, 480875), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:13:44,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:13:44,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:13:44,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:13:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:13:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:00.978909+00:00 (in 13.790958 seconds) exportactionlogsworker stdout | 2022-05-05 10:13:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:47 UTC)" (scheduled at 2022-05-05 10:13:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:13:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:13:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:14:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:13:47,369 [219] [DEBUG] [app] Starting request: urn:request:83c62dfa-f410-4781-9aa9-32cfb6b8c727 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:47,369 [221] [DEBUG] [app] Starting request: urn:request:3d82af9b-dde7-4067-bca7-d56ca14ab99f (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:47,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:47,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:13:47,373 [224] [DEBUG] [app] Starting request: urn:request:aa0b2261-cec8-47ca-bbe1-a0a29f7943f8 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:47,373 [226] [DEBUG] [app] Starting request: urn:request:79825b6d-61dd-4e1a-b6a1-8bd0d636e7d2 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:47,374 [224] [DEBUG] [app] Ending request: urn:request:aa0b2261-cec8-47ca-bbe1-a0a29f7943f8 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:13:47,374 [226] [DEBUG] [app] Ending request: urn:request:79825b6d-61dd-4e1a-b6a1-8bd0d636e7d2 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:13:47,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:13:47,374 [226] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:47,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:47,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:47,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:13:47,378 [219] [DEBUG] [app] Starting request: urn:request:ccfc2033-e382-4f66-aec5-d0ee7fb46583 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:47,378 [222] [DEBUG] [app] Starting request: urn:request:67f6e2ba-3195-471c-afa2-9c98f54ac206 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:47,379 [222] [DEBUG] [app] Ending request: urn:request:67f6e2ba-3195-471c-afa2-9c98f54ac206 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:13:47,379 [219] [DEBUG] [app] Ending request: urn:request:ccfc2033-e382-4f66-aec5-d0ee7fb46583 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:13:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:13:47,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:47,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:13:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:13:47,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:47,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:13:47,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:13:47,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:13:47,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:13:47,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:13:47,394 [219] [DEBUG] [app] Ending request: urn:request:83c62dfa-f410-4781-9aa9-32cfb6b8c727 (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:47,394 [221] [DEBUG] [app] Ending request: urn:request:3d82af9b-dde7-4067-bca7-d56ca14ab99f (/health/instance) gunicorn-web stdout | 2022-05-05 10:13:47,394 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:13:47,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:13:47,395 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:13:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:13:47,395 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:13:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:13:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:13:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.026) gunicorn-registry stdout | 2022-05-05 10:13:50,947 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:13:50,951 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:13:50,959 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:13:50,982 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:13:50,987 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:13:50,995 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:13:51,002 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:13:51,006 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} gunicorn-registry stdout | 2022-05-05 10:13:51,510 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} notificationworker stdout | 2022-05-05 10:13:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:13:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:03.089985+00:00 (in 9.999417 seconds) notificationworker stdout | 2022-05-05 10:13:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:03 UTC)" (scheduled at 2022-05-05 10:13:53.089985+00:00) notificationworker stdout | 2022-05-05 10:13:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:13:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 13, 53, 91085), True, datetime.datetime(2022, 5, 5, 10, 13, 53, 91085), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:13:53,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:13:53,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:13:53,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:13:55,622 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:13:56,213 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:13:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:13:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:26.865082+00:00 (in 29.999389 seconds) gcworker stdout | 2022-05-05 10:13:56,866 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:26 UTC)" (scheduled at 2022-05-05 10:13:56.865082+00:00) gcworker stdout | 2022-05-05 10:13:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536036866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:13:56,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:13:56,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:13:56,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:13:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:13:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:28.201650+00:00 (in 29.999530 seconds) securityworker stdout | 2022-05-05 10:13:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:28 UTC)" (scheduled at 2022-05-05 10:13:58.201650+00:00) securityworker stdout | 2022-05-05 10:13:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:13:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:13:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:13:58,208 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:13:58,217 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:13:58,217 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:14:00,009 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:14:00,158 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:14:00,512 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:14:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:14:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:47.187465+00:00 (in 46.207962 seconds) exportactionlogsworker stdout | 2022-05-05 10:14:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:00 UTC)" (scheduled at 2022-05-05 10:14:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:14:00,980 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:14:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 0, 980076), True, datetime.datetime(2022, 5, 5, 10, 14, 0, 980076), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:14:00,990 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:14:00,990 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:14:00,990 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:00 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:14:01,079 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:14:01,443 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:14:01,754 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:14:02,369 [219] [DEBUG] [app] Starting request: urn:request:0358bb99-01f8-4085-b6bd-71625750d105 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:02,369 [221] [DEBUG] [app] Starting request: urn:request:298ede7d-b699-47d6-9203-28f46a05a2fe (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:02,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:02,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:14:02,373 [224] [DEBUG] [app] Starting request: urn:request:6d021644-145a-4db0-93e8-c853e237049f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:02,373 [231] [DEBUG] [app] Starting request: urn:request:f724b441-92c2-470d-8efe-83bcd14d7e14 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:02,374 [231] [DEBUG] [app] Ending request: urn:request:f724b441-92c2-470d-8efe-83bcd14d7e14 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:02,374 [224] [DEBUG] [app] Ending request: urn:request:6d021644-145a-4db0-93e8-c853e237049f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:02,374 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:14:02,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:02,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:14:02,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:02,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:02,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:02,378 [222] [DEBUG] [app] Starting request: urn:request:0f5b5300-c09f-4e14-9efe-2c2e50704c08 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:02,379 [222] [DEBUG] [app] Ending request: urn:request:0f5b5300-c09f-4e14-9efe-2c2e50704c08 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:02,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:02,379 [219] [DEBUG] [app] Starting request: urn:request:032ea3a5-551d-4422-91ea-2b680ef45df2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:02,380 [219] [DEBUG] [app] Ending request: urn:request:032ea3a5-551d-4422-91ea-2b680ef45df2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:02,381 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 159 0.003) gunicorn-web stdout | 2022-05-05 10:14:02,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:02,381 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:02,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:14:02,382 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:14:02,389 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:14:02,389 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:14:02,396 [219] [DEBUG] [app] Ending request: urn:request:0358bb99-01f8-4085-b6bd-71625750d105 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:02,396 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:02,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:14:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:14:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-web stdout | 2022-05-05 10:14:02,397 [221] [DEBUG] [app] Ending request: urn:request:298ede7d-b699-47d6-9203-28f46a05a2fe (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:02,398 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:02,398 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:14:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:14:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.029) gunicorn-secscan stdout | 2022-05-05 10:14:03,032 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:14:03,035 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:14:03,042 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:14:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:14:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:12.207990+00:00 (in 9.117504 seconds) notificationworker stdout | 2022-05-05 10:14:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:13 UTC)" (scheduled at 2022-05-05 10:14:03.089985+00:00) notificationworker stdout | 2022-05-05 10:14:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:14:03,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 3, 90914), True, datetime.datetime(2022, 5, 5, 10, 14, 3, 90914), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:14:03,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:14:03,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:14:03,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:13 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-05 10:14:03,126 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:14:03,223 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:14:03,625 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:14:03,767 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:14:03,908 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:14:04,623 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:14:05,647 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:14:05,807 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:14:05,874 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:14:06,049 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:14:06,062 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:14:06,066 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:14:06,072 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:14:06,075 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:14:06,108 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:14:06,392 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:14:07,269 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:14:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:14:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:13.089985+00:00 (in 0.881491 seconds) notificationworker stdout | 2022-05-05 10:14:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:12 UTC)" (scheduled at 2022-05-05 10:14:12.207990+00:00) notificationworker stdout | 2022-05-05 10:14:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:14:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:14:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:14:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:23.089985+00:00 (in 9.999505 seconds) notificationworker stdout | 2022-05-05 10:14:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:23 UTC)" (scheduled at 2022-05-05 10:14:13.089985+00:00) notificationworker stdout | 2022-05-05 10:14:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:14:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 13, 90964), True, datetime.datetime(2022, 5, 5, 10, 14, 13, 90964), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:14:13,100 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:14:13,100 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:14:13,100 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:14:13,394 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:14:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:24.984990+00:00 (in 11.589903 seconds) repositorygcworker stdout | 2022-05-05 10:14:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:13 UTC)" (scheduled at 2022-05-05 10:14:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:14:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:14:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:14:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:14:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:23.280939+00:00 (in 8.993961 seconds) namespacegcworker stdout | 2022-05-05 10:14:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:14 UTC)" (scheduled at 2022-05-05 10:14:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:14:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:14:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:14:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:14:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:44.479974+00:00 (in 29.999488 seconds) buildlogsarchiver stdout | 2022-05-05 10:14:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:44 UTC)" (scheduled at 2022-05-05 10:14:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:14:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 14, 14, 480998), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:14:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:14:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:14:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:14:17,370 [220] [DEBUG] [app] Starting request: urn:request:b1998a48-4094-4fcc-aac1-9d313933f5a3 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:17,370 [219] [DEBUG] [app] Starting request: urn:request:f839ff8a-7e72-4bb7-a2a1-357959c043fa (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:17,372 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:17,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:14:17,374 [228] [DEBUG] [app] Starting request: urn:request:67209c56-0065-4871-833d-40d7b2270fcc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:17,374 [231] [DEBUG] [app] Starting request: urn:request:f5c38125-e0cb-4eba-b1b7-9e3363584bd8 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:17,374 [228] [DEBUG] [app] Ending request: urn:request:67209c56-0065-4871-833d-40d7b2270fcc (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:17,375 [231] [DEBUG] [app] Ending request: urn:request:f5c38125-e0cb-4eba-b1b7-9e3363584bd8 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:14:17,375 [228] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:17,375 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.004 162 0.005) gunicorn-registry stdout | 2022-05-05 10:14:17,377 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:17,378 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:17,378 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:17,379 [222] [DEBUG] [app] Starting request: urn:request:8adf8833-a211-4e8a-abf8-a50dc43b2e19 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:17,380 [222] [DEBUG] [app] Ending request: urn:request:8adf8833-a211-4e8a-abf8-a50dc43b2e19 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:17,380 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:17,380 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:17,381 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:14:17,381 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:14:17,382 [221] [DEBUG] [app] Starting request: urn:request:dfea9732-59ce-4dc1-a2ae-e39925c1b19a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:17,382 [221] [DEBUG] [app] Ending request: urn:request:dfea9732-59ce-4dc1-a2ae-e39925c1b19a (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:14:17,383 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:17,383 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:17,384 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:14:17,389 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:14:17,389 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:14:17,396 [219] [DEBUG] [app] Ending request: urn:request:f839ff8a-7e72-4bb7-a2a1-357959c043fa (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:17,396 [220] [DEBUG] [app] Ending request: urn:request:b1998a48-4094-4fcc-aac1-9d313933f5a3 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:17,396 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:17,396 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:17,397 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:14:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:14:17,397 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:14:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:14:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:14:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-registry stdout | 2022-05-05 10:14:20,960 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:14:20,964 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:14:20,972 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:14:20,992 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:14:20,999 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:14:21,004 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:14:21,013 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} gunicorn-registry stdout | 2022-05-05 10:14:21,017 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:14:21,852 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} notificationworker stdout | 2022-05-05 10:14:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:14:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:33.089985+00:00 (in 9.999516 seconds) notificationworker stdout | 2022-05-05 10:14:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:33 UTC)" (scheduled at 2022-05-05 10:14:23.089985+00:00) notificationworker stdout | 2022-05-05 10:14:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:14:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 23, 90977), True, datetime.datetime(2022, 5, 5, 10, 14, 23, 90977), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:14:23,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:14:23,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:14:23,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:33 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:14:23,281 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:14:23,281 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:38.977973+00:00 (in 15.696548 seconds) namespacegcworker stdout | 2022-05-05 10:14:23,281 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:19:23 UTC)" (scheduled at 2022-05-05 10:14:23.280939+00:00) namespacegcworker stdout | 2022-05-05 10:14:23,282 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 14, 23, 281891), 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:14:23,293 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 14, 23, 281891), True, datetime.datetime(2022, 5, 5, 10, 14, 23, 281891), 0, 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:14:23,296 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 14, 23, 281891), True, datetime.datetime(2022, 5, 5, 10, 14, 23, 281891), 0, 'namespacegc/%', False, datetime.datetime(2022, 5, 5, 10, 14, 23, 281891), 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:14:23,299 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:14:23,299 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:19:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:14:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:14:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:13.394619+00:00 (in 48.409172 seconds) repositorygcworker stdout | 2022-05-05 10:14:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:24 UTC)" (scheduled at 2022-05-05 10:14:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:14:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:14:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 24, 985978), True, datetime.datetime(2022, 5, 5, 10, 14, 24, 985978), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:14:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:14:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:14:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:14:25,661 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:14:26,252 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:14:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:14:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:56.865082+00:00 (in 29.999487 seconds) gcworker stdout | 2022-05-05 10:14:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:56 UTC)" (scheduled at 2022-05-05 10:14:26.865082+00:00) gcworker stdout | 2022-05-05 10:14:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536066866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:14:26,886 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:14:26,886 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:14:26,886 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:56 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:14:28,064 [222] [DEBUG] [app] Starting request: urn:request:375355bc-257d-4938-9b7e-d07e6fa8138a (/api/v1/user/notifications) gunicorn-web stdout | 2022-05-05 10:14:28,064 [222] [DEBUG] [app] User loader loading deferred user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:14:28,065 [222] [DEBUG] [auth.cookie] Loading user from cookie: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:14:28,065 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['105a42ac-0110-4aed-8579-1c532997244a', False, 1, 0]) gunicorn-web stdout | 2022-05-05 10:14:28,076 [222] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-05 10:14:28,076 [222] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-05 10:14:28,076 [222] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:14:28,076 [222] [DEBUG] [endpoints.api] Checking permission for user ibazulic gunicorn-web stdout | 2022-05-05 10:14:28,076 [222] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:14:28,076 [222] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:14:28,076 [222] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:14:28,076 [222] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:14:28,078 [222] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2022-05-05 10:14:28,084 [222] [DEBUG] [app] Ending request: urn:request:375355bc-257d-4938-9b7e-d07e6fa8138a (/api/v1/user/notifications) gunicorn-web stdout | 2022-05-05 10:14:28,085 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:28,085 [222] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:14:28 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:14:28 +0000] "GET /api/v1/user/notifications HTTP/1.1" 200 43 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" (0.022 1294 0.022) gunicorn-web stdout | 2022-05-05 10:14:28,095 [222] [DEBUG] [app] Starting request: urn:request:def3db92-fe9a-4df9-830c-f5211b7ef9a4 (/api/v1/user/apptoken) gunicorn-web stdout | 2022-05-05 10:14:28,095 [222] [DEBUG] [app] User loader loading deferred user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:14:28,096 [222] [DEBUG] [auth.cookie] Loading user from cookie: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:14:28,097 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['105a42ac-0110-4aed-8579-1c532997244a', False, 1, 0]) gunicorn-web stdout | 2022-05-05 10:14:28,106 [222] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-05 10:14:28,106 [222] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-05 10:14:28,106 [222] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:14:28,106 [222] [DEBUG] [endpoints.api] Checking permission for user ibazulic gunicorn-web stdout | 2022-05-05 10:14:28,106 [222] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:14:28,106 [222] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:14:28,106 [222] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:14:28,106 [222] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:14:28,107 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2022, 5, 8, 5, 26, 28, 106894), datetime.datetime(2022, 5, 5, 10, 14, 28, 106961)]) gunicorn-web stdout | 2022-05-05 10:14:28,111 [222] [DEBUG] [app] Ending request: urn:request:def3db92-fe9a-4df9-830c-f5211b7ef9a4 (/api/v1/user/apptoken) gunicorn-web stdout | 2022-05-05 10:14:28,112 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:28,112 [222] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:14:28 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:14:28 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.1" 200 38 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" (0.018 1303 0.018) securityworker stdout | 2022-05-05 10:14:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:14:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:58.201650+00:00 (in 29.999537 seconds) securityworker stdout | 2022-05-05 10:14:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:58 UTC)" (scheduled at 2022-05-05 10:14:28.201650+00:00) securityworker stdout | 2022-05-05 10:14:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:14:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:14:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:14:28,208 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:14:28,218 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:14:28,220 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:14:28,223 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:14:28,223 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:14:28,224 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 9, 28, 223496), 1, 2]) securityworker stdout | 2022-05-05 10:14:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:14:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:14:28,226 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 9, 28, 225976), 1, 2]) securityworker stdout | 2022-05-05 10:14:28,228 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:14:28,229 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:14:28,229 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:14:28,229 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:14:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:14:30,033 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:14:30,195 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:14:30,542 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:14:31,117 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:14:31,480 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} securityscanningnotificationworker stdout | 2022-05-05 10:14:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:14:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:35.969483+00:00 (in 4.194971 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:14:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:31 UTC)" (scheduled at 2022-05-05 10:14:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:14:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:14:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 31, 775301), True, datetime.datetime(2022, 5, 5, 10, 14, 31, 775301), 0, 'secscanv4/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:14:31,779 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} securityscanningnotificationworker stdout | 2022-05-05 10:14:31,787 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:14:31,787 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:14:31,787 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:31 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:14:32,369 [219] [DEBUG] [app] Starting request: urn:request:38e3daa4-0842-48dd-bc28-62214932dbc8 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:32,369 [220] [DEBUG] [app] Starting request: urn:request:80fd05a4-424b-43f1-9112-f74dc9b37405 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:32,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:32,371 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:14:32,373 [224] [DEBUG] [app] Starting request: urn:request:710e4d50-048d-45b5-ae7d-4e61fd60fd77 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:32,373 [230] [DEBUG] [app] Starting request: urn:request:124a79c9-d0bb-4577-9f07-18c08445a8f7 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:32,374 [224] [DEBUG] [app] Ending request: urn:request:710e4d50-048d-45b5-ae7d-4e61fd60fd77 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:32,374 [230] [DEBUG] [app] Ending request: urn:request:124a79c9-d0bb-4577-9f07-18c08445a8f7 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:14:32,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:14:32,374 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:32,374 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:32,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:32,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:32,376 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:32,378 [219] [DEBUG] [app] Starting request: urn:request:505340ab-9294-4a5f-a288-057d21990db1 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:32,378 [222] [DEBUG] [app] Starting request: urn:request:75283036-5c5b-4c1d-8bf3-458c868021a8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:32,378 [219] [DEBUG] [app] Ending request: urn:request:505340ab-9294-4a5f-a288-057d21990db1 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:32,379 [222] [DEBUG] [app] Ending request: urn:request:75283036-5c5b-4c1d-8bf3-458c868021a8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:32,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:14:32,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:32,379 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:32,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:14:32,380 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:14:32,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:14:32,386 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:14:32,390 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:14:32,393 [219] [DEBUG] [app] Ending request: urn:request:38e3daa4-0842-48dd-bc28-62214932dbc8 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:32,393 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:32,394 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:14:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:14:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:14:32,397 [220] [DEBUG] [app] Ending request: urn:request:80fd05a4-424b-43f1-9112-f74dc9b37405 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:32,397 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:32,397 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:14:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:14:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.030) gunicorn-secscan stdout | 2022-05-05 10:14:33,041 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:14:33,045 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:14:33,050 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:14:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:14:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:43.089985+00:00 (in 9.999498 seconds) notificationworker stdout | 2022-05-05 10:14:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:43 UTC)" (scheduled at 2022-05-05 10:14:33.089985+00:00) notificationworker stdout | 2022-05-05 10:14:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:14:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 33, 91081), True, datetime.datetime(2022, 5, 5, 10, 14, 33, 91081), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:14:33,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:14:33,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:14:33,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:43 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-05 10:14:33,160 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:14:33,261 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:14:33,651 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:14:33,776 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:14:33,942 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:14:34,659 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:14:35,668 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:14:35,845 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:14:35,899 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} securityscanningnotificationworker stdout | 2022-05-05 10:14:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:14:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:31.774027+00:00 (in 55.803988 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:14:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:35 UTC)" (scheduled at 2022-05-05 10:14:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:14:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:14:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:35 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:14:36,057 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:14:36,071 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:14:36,079 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:14:36,083 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:14:36,087 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:14:36,124 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:14:36,428 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:14:37,302 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:14:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:14:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:14.286471+00:00 (in 35.307979 seconds) namespacegcworker stdout | 2022-05-05 10:14:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:38 UTC)" (scheduled at 2022-05-05 10:14:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:14:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:14:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 38, 979045), True, datetime.datetime(2022, 5, 5, 10, 14, 38, 979045), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:14:38,991 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:14:38,991 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:14:38,991 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:38 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request URL: 'https://quaystorageaccount2.blob.core.windows.net/quayblobstorage/registry/uploads/5a78111c-05a1-4b40-9160-2cbb59d6743d?blockid=REDACTED&comp=REDACTED' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request method: 'PUT' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] Request headers: gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Type': 'application/octet-stream' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-Length': '964328669' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-version': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-date': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'x-ms-client-request-id': '2c08317e-cc5c-11ec-a67b-0a580a82021b' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'User-Agent': 'azsdk-python-storage-blob/12.4.0 Python/3.8.8 (Linux-4.18.0-305.40.2.el8_4.x86_64-x86_64-with-glibc2.2.5)' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Content-MD5': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] 'Authorization': 'REDACTED' gunicorn-registry stdout | 2022-05-05 10:14:40,804 [224] [INFO] [azure.core.pipeline.policies.http_logging_policy] A body is sent with the request gunicorn-registry stdout | 2022-05-05 10:14:40,810 [224] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: quaystorageaccount2.blob.core.windows.net notificationworker stdout | 2022-05-05 10:14:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:14:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:14:53.089985+00:00 (in 9.999462 seconds) notificationworker stdout | 2022-05-05 10:14:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:53 UTC)" (scheduled at 2022-05-05 10:14:43.089985+00:00) notificationworker stdout | 2022-05-05 10:14:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:14:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 43, 91085), True, datetime.datetime(2022, 5, 5, 10, 14, 43, 91085), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:14:43,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:14:43,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:14:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:14:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:14:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:14:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:14.479974+00:00 (in 29.999471 seconds) buildlogsarchiver stdout | 2022-05-05 10:14:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:14 UTC)" (scheduled at 2022-05-05 10:14:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:14:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 14, 44, 480966), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:14:44,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:14:44,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:14:44,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:14:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:14:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:00.978909+00:00 (in 13.790925 seconds) exportactionlogsworker stdout | 2022-05-05 10:14:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:47 UTC)" (scheduled at 2022-05-05 10:14:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:14:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:14:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:15:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:14:47,369 [219] [DEBUG] [app] Starting request: urn:request:f063a115-0546-4239-82c5-092ec4429df6 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:47,369 [221] [DEBUG] [app] Starting request: urn:request:9309a7fa-cc6e-4f93-b5b3-71438c93505c (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:47,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:47,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:14:47,373 [229] [DEBUG] [app] Starting request: urn:request:d1079384-85e8-45b7-9be8-1cc76ea376bd (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:47,373 [224] [DEBUG] [app] Starting request: urn:request:c6b06a83-9810-49ac-bfa1-5961b429ecbf (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:47,373 [229] [DEBUG] [app] Ending request: urn:request:d1079384-85e8-45b7-9be8-1cc76ea376bd (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:14:47,374 [224] [DEBUG] [app] Ending request: urn:request:c6b06a83-9810-49ac-bfa1-5961b429ecbf (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-registry stdout | 2022-05-05 10:14:47,374 [229] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:14:47,374 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:47,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:47,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:47,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:14:47,378 [219] [DEBUG] [app] Starting request: urn:request:c4aa5268-8f67-40a5-8865-3e62512687ab (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:47,378 [222] [DEBUG] [app] Starting request: urn:request:cc8478a1-4a0a-472f-91f7-d5259752b83d (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:47,378 [219] [DEBUG] [app] Ending request: urn:request:c4aa5268-8f67-40a5-8865-3e62512687ab (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:47,379 [222] [DEBUG] [app] Ending request: urn:request:cc8478a1-4a0a-472f-91f7-d5259752b83d (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:14:47,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:14:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:14:47,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:47,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:14:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:14:47,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:14:47,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:14:47,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:14:47,386 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:14:47,386 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:14:47,392 [221] [DEBUG] [app] Ending request: urn:request:9309a7fa-cc6e-4f93-b5b3-71438c93505c (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:47,393 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:47,393 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:14:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:14:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:14:47,394 [219] [DEBUG] [app] Ending request: urn:request:f063a115-0546-4239-82c5-092ec4429df6 (/health/instance) gunicorn-web stdout | 2022-05-05 10:14:47,394 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:14:47,395 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:14:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:14:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-registry stdout | 2022-05-05 10:14:50,975 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:14:50,978 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:14:50,984 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:14:51,004 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:14:51,011 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:14:51,014 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:14:51,025 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} gunicorn-registry stdout | 2022-05-05 10:14:51,029 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:14:52,461 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} notificationworker stdout | 2022-05-05 10:14:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:14:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:03.089985+00:00 (in 9.999488 seconds) notificationworker stdout | 2022-05-05 10:14:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:03 UTC)" (scheduled at 2022-05-05 10:14:53.089985+00:00) notificationworker stdout | 2022-05-05 10:14:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:14:53,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 14, 53, 90989), True, datetime.datetime(2022, 5, 5, 10, 14, 53, 90989), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:14:53,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:14:53,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:14:53,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:14:55,697 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:14:56,289 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:14:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:14:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:26.865082+00:00 (in 29.999512 seconds) gcworker stdout | 2022-05-05 10:14:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:26 UTC)" (scheduled at 2022-05-05 10:14:56.865082+00:00) gcworker stdout | 2022-05-05 10:14:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536096866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:14:56,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:14:56,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:14:56,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:14:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:14:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:28.201650+00:00 (in 29.999500 seconds) securityworker stdout | 2022-05-05 10:14:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:28 UTC)" (scheduled at 2022-05-05 10:14:58.201650+00:00) securityworker stdout | 2022-05-05 10:14:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:14:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:14:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:14:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:14:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:14:58,217 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:15:00,053 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:15:00,231 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:15:00,583 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:15:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:15:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:47.187465+00:00 (in 46.208042 seconds) exportactionlogsworker stdout | 2022-05-05 10:15:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:00 UTC)" (scheduled at 2022-05-05 10:15:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:15:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:15:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 0, 979963), True, datetime.datetime(2022, 5, 5, 10, 15, 0, 979963), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:15:00,992 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:15:00,992 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:15:00,992 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:00 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:15:01,154 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:15:01,518 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:15:01,813 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:15:02,370 [222] [DEBUG] [app] Starting request: urn:request:a1a1231c-5d43-4d3a-9d0e-685252ad5c9f (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:02,370 [219] [DEBUG] [app] Starting request: urn:request:e6ea55bd-bbd5-4996-87d9-e2343c910db6 (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:02,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:02,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:15:02,374 [230] [DEBUG] [app] Starting request: urn:request:bdc8ff78-85d1-402e-a858-dd3b32d2e3c5 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:02,374 [224] [DEBUG] [app] Starting request: urn:request:db11abf1-45f8-4ab4-83b8-08e796943159 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:02,374 [224] [DEBUG] [app] Ending request: urn:request:db11abf1-45f8-4ab4-83b8-08e796943159 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:02,374 [230] [DEBUG] [app] Ending request: urn:request:bdc8ff78-85d1-402e-a858-dd3b32d2e3c5 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:02,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:15:02,375 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:15:02,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:02,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:15:02,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:02,378 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:02,379 [222] [DEBUG] [app] Starting request: urn:request:737be518-c557-4169-9f92-9deabee39f9c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:02,380 [222] [DEBUG] [app] Ending request: urn:request:737be518-c557-4169-9f92-9deabee39f9c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:02,380 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:15:02,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:02,380 [219] [DEBUG] [app] Starting request: urn:request:cf531e4b-a62b-4f53-912c-24dff69b737b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:02,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:15:02,381 [219] [DEBUG] [app] Ending request: urn:request:cf531e4b-a62b-4f53-912c-24dff69b737b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:02,381 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:15:02,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:15:02,383 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:15:02,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:15:02,389 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:15:02,394 [222] [DEBUG] [app] Ending request: urn:request:a1a1231c-5d43-4d3a-9d0e-685252ad5c9f (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:02,394 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:15:02,395 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:15:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:15:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:15:02,396 [219] [DEBUG] [app] Ending request: urn:request:e6ea55bd-bbd5-4996-87d9-e2343c910db6 (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:02,396 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:15:02,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:15:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:15:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.028) gunicorn-secscan stdout | 2022-05-05 10:15:03,053 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:15:03,057 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:15:03,061 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:15:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:15:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:12.207990+00:00 (in 9.117504 seconds) notificationworker stdout | 2022-05-05 10:15:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:13 UTC)" (scheduled at 2022-05-05 10:15:03.089985+00:00) notificationworker stdout | 2022-05-05 10:15:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:15:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 3, 91057), True, datetime.datetime(2022, 5, 5, 10, 15, 3, 91057), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:15:03,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:15:03,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:15:03,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:13 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-05 10:15:03,198 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:15:03,299 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:15:03,673 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:15:03,814 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:15:03,979 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:15:04,697 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:15:05,705 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:15:05,882 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:15:05,938 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:15:06,066 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:15:06,080 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:15:06,091 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:15:06,097 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:15:06,103 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:15:06,161 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:15:06,467 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:15:07,338 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} nginx stdout | 10.130.2.11 () - - [05/May/2022:10:15:07 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/5a78111c-05a1-4b40-9160-2cbb59d6743d HTTP/1.1" 499 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (295.210 964950573 -) notificationworker stdout | 2022-05-05 10:15:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:15:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:13.089985+00:00 (in 0.881496 seconds) notificationworker stdout | 2022-05-05 10:15:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:12 UTC)" (scheduled at 2022-05-05 10:15:12.207990+00:00) notificationworker stdout | 2022-05-05 10:15:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:15:12,208 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:12 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:15:12,857 [224] [DEBUG] [app] Starting request: urn:request:15649428-9244-44e7-9f40-b734a52f31ce (/v2/auth) gunicorn-registry stdout | 2022-05-05 10:15:12,858 [224] [DEBUG] [auth.basic] Attempt to process basic auth header gunicorn-registry stdout | 2022-05-05 10:15:12,858 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."username" = %s) OR ("t1"."email" = %s)) LIMIT %s OFFSET %s', ['ibazulic', 'ibazulic', 1, 0]) notificationworker stdout | 2022-05-05 10:15:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:15:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:23.089985+00:00 (in 9.999527 seconds) notificationworker stdout | 2022-05-05 10:15:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:23 UTC)" (scheduled at 2022-05-05 10:15:13.089985+00:00) notificationworker stdout | 2022-05-05 10:15:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:15:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 13, 90983), True, datetime.datetime(2022, 5, 5, 10, 15, 13, 90983), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:15:13,104 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:15:13,104 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:15:13,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:23 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:15:13,184 [224] [DEBUG] [auth.credentials] Successfully validated credentials for user ibazulic gunicorn-registry stdout | 2022-05-05 10:15:13,184 [224] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-registry stdout | 2022-05-05 10:15:13,184 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:15:13,184 [224] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-registry stdout | 2022-05-05 10:15:13,185 [224] [DEBUG] [endpoints.v2.v2auth] Request audience: quay-quay-quay-enterprise.apps.rcegan.azemea.com gunicorn-registry stdout | 2022-05-05 10:15:13,185 [224] [DEBUG] [endpoints.v2.v2auth] Scope request: ['repository:ibazulic/test-big-layers:push,pull'] gunicorn-registry stdout | 2022-05-05 10:15:13,185 [224] [DEBUG] [endpoints.v2.v2auth] Match: ('ibazulic/test-big-layers', 'ibazulic/test-big-layers', 'push,pull') gunicorn-registry stdout | 2022-05-05 10:15:13,186 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE ("t1"."username" = %s) LIMIT %s OFFSET %s', ['ibazulic', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:15:13,189 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:15:13,191 [224] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-registry stdout | 2022-05-05 10:15:13,191 [224] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-registry stdout | 2022-05-05 10:15:13,191 [224] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-registry stdout | 2022-05-05 10:15:13,191 [224] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-registry stdout | 2022-05-05 10:15:13,191 [224] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-registry stdout | 2022-05-05 10:15:13,199 [224] [DEBUG] [app] Ending request: urn:request:15649428-9244-44e7-9f40-b734a52f31ce (/v2/auth) gunicorn-registry stdout | 2022-05-05 10:15:13,199 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:15:13,199 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:15:13,199 [224] [INFO] [gunicorn.access] 10.128.2.6 - ibazulic [05/May/2022:10:15:13 +0000] "GET /v2/auth?account=ibazulic&scope=repository%3Aibazulic%2Ftest-big-layers%3Apush%2Cpull&service=quay-quay-quay-enterprise.apps.rcegan.azemea.com HTTP/1.1" 200 1081 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.128.2.6 () - ibazulic [05/May/2022:10:15:13 +0000] "GET /v2/auth?account=ibazulic&scope=repository%3Aibazulic%2Ftest-big-layers%3Apush%2Cpull&service=quay-quay-quay-enterprise.apps.rcegan.azemea.com HTTP/1.1" 200 1081 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.342 683 0.343) gunicorn-registry stdout | 2022-05-05 10:15:13,216 [224] [DEBUG] [data.userevent] Published user event docker-cli: {'action': 'push_start', 'namespace': 'ibazulic', 'repository': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:15:13,332 [224] [DEBUG] [app] Starting request: urn:request:caea93a8-4ccd-431e-90d8-e722ffeebe60 (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:15:13,333 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:15:13,333 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NzEzLCJpYXQiOjE2NTE3NDU3MTMsImV4cCI6MTY1MTc0OTMxMywic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.ggV3fetHHKAme7s_4l2wMBhcBigRGJG12HVVbE9Dw_pwxWzlbgMaLSk8PELONxAz5cdPyt5cQNf5DrW_-4aBpaISFnyZMjwc7lTndZOjK1iPw2jgOX9JuTvtWMEVh6iXPRm1SZeY2j9UIhjGYdF-FOsy9uazuS0-8DyJF9LuPlWQychfiHgsEkXpBPoxpJ5VuuRDAuxxgJrYlTsNMS05CvcR18t_fx9FxMEE2VCtjMwRswlBGgznwVXQo0n1YMxW3otNDWvnYJkbGjtTs_PE-5GqdLJCIYw8GcavjWwto6RDNO8_dakeUE7t68fRxmRhGFa1YDSKyIAPQg6wyANkiQ gunicorn-registry stdout | 2022-05-05 10:15:13,333 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NzEzLCJpYXQiOjE2NTE3NDU3MTMsImV4cCI6MTY1MTc0OTMxMywic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.ggV3fetHHKAme7s_4l2wMBhcBigRGJG12HVVbE9Dw_pwxWzlbgMaLSk8PELONxAz5cdPyt5cQNf5DrW_-4aBpaISFnyZMjwc7lTndZOjK1iPw2jgOX9JuTvtWMEVh6iXPRm1SZeY2j9UIhjGYdF-FOsy9uazuS0-8DyJF9LuPlWQychfiHgsEkXpBPoxpJ5VuuRDAuxxgJrYlTsNMS05CvcR18t_fx9FxMEE2VCtjMwRswlBGgznwVXQo0n1YMxW3otNDWvnYJkbGjtTs_PE-5GqdLJCIYw8GcavjWwto6RDNO8_dakeUE7t68fRxmRhGFa1YDSKyIAPQg6wyANkiQ gunicorn-registry stdout | 2022-05-05 10:15:13,334 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:15:13,334 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:15:13,334 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:15:13,334 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:15:13,334 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:15:13,336 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:15:13,339 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state" FROM "repository" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s', [2, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:15:13,341 [224] [DEBUG] [peewee] ('INSERT INTO "blobupload" ("repository_id", "uuid", "byte_count", "sha_state", "location_id", "storage_metadata", "chunk_count", "created") VALUES (%s, %s, %s, %s, %s, %s, %s, %s) RETURNING "blobupload"."id"', [2, '8c27a1e5-8725-40a2-a184-f4e11fe5bc86', 0, 'gASVqwAAAAAAAACMBnJlaGFzaJSMBnNoYTI1NpSTlCmBlH2UKIwEbmFtZZRoAYwHbWRfZGF0YZRDeGfmCWqFrme7cvNuPDr1T6V/Ug5RjGgFm6vZgx8ZzeBbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAJR1Yi4=', 9, '{"blocks": [], "content-type": null}', 0, datetime.datetime(2022, 5, 5, 10, 15, 13, 341030)]) gunicorn-registry stdout | 2022-05-05 10:15:13,345 [224] [DEBUG] [app] Ending request: urn:request:caea93a8-4ccd-431e-90d8-e722ffeebe60 (/v2/ibazulic/test-big-layers/blobs/uploads/) gunicorn-registry stdout | 2022-05-05 10:15:13,345 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:15:13,345 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. gunicorn-registry stdout | 2022-05-05 10:15:13,345 [224] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:15:13 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \(linux\))" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:15:13 +0000] "POST /v2/ibazulic/test-big-layers/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (0.013 1649 0.014) repositorygcworker stdout | 2022-05-05 10:15:13,394 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:15:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:24.984990+00:00 (in 11.589897 seconds) repositorygcworker stdout | 2022-05-05 10:15:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:13 UTC)" (scheduled at 2022-05-05 10:15:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:15:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:15:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:13 UTC)" executed successfully gunicorn-registry stdout | 2022-05-05 10:15:13,527 [224] [DEBUG] [app] Starting request: urn:request:fa3d5430-67a3-4ed5-b4ef-a5ec6af8da1c (/v2/ibazulic/test-big-layers/blobs/uploads/8c27a1e5-8725-40a2-a184-f4e11fe5bc86) gunicorn-registry stdout | 2022-05-05 10:15:13,527 [224] [DEBUG] [auth.registry_jwt_auth] Called with params: (), {'upload_uuid': '8c27a1e5-8725-40a2-a184-f4e11fe5bc86', 'namespace_name': 'ibazulic', 'repo_name': 'test-big-layers'} gunicorn-registry stdout | 2022-05-05 10:15:13,527 [224] [DEBUG] [auth.registry_jwt_auth] Validating auth header: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NzEzLCJpYXQiOjE2NTE3NDU3MTMsImV4cCI6MTY1MTc0OTMxMywic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.ggV3fetHHKAme7s_4l2wMBhcBigRGJG12HVVbE9Dw_pwxWzlbgMaLSk8PELONxAz5cdPyt5cQNf5DrW_-4aBpaISFnyZMjwc7lTndZOjK1iPw2jgOX9JuTvtWMEVh6iXPRm1SZeY2j9UIhjGYdF-FOsy9uazuS0-8DyJF9LuPlWQychfiHgsEkXpBPoxpJ5VuuRDAuxxgJrYlTsNMS05CvcR18t_fx9FxMEE2VCtjMwRswlBGgznwVXQo0n1YMxW3otNDWvnYJkbGjtTs_PE-5GqdLJCIYw8GcavjWwto6RDNO8_dakeUE7t68fRxmRhGFa1YDSKyIAPQg6wyANkiQ gunicorn-registry stdout | 2022-05-05 10:15:13,527 [224] [DEBUG] [util.security.registry_jwt] encoded JWT: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IkktNEl6bDhrWnJ1WjRpdnZKLTFFLUJhWVN3ZTZkX3FYREVtWnZGT0pJbzAifQ.eyJpc3MiOiJxdWF5IiwiYXVkIjoicXVheS1xdWF5LXF1YXktZW50ZXJwcmlzZS5hcHBzLnJjZWdhbi5hemVtZWEuY29tIiwibmJmIjoxNjUxNzQ1NzEzLCJpYXQiOjE2NTE3NDU3MTMsImV4cCI6MTY1MTc0OTMxMywic3ViIjoiaWJhenVsaWMiLCJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImliYXp1bGljL3Rlc3QtYmlnLWxheWVycyIsImFjdGlvbnMiOlsicHVzaCIsInB1bGwiXX1dLCJjb250ZXh0Ijp7InZlcnNpb24iOjIsImVudGl0eV9raW5kIjoidXNlciIsImVudGl0eV9yZWZlcmVuY2UiOiIxMDVhNDJhYy0wMTEwLTRhZWQtODU3OS0xYzUzMjk5NzI0NGEiLCJraW5kIjoidXNlciIsInVzZXIiOiJpYmF6dWxpYyIsImNvbS5hcG9zdGlsbGUucm9vdHMiOnsiaWJhenVsaWMvdGVzdC1iaWctbGF5ZXJzIjoiJGRpc2FibGVkIn0sImNvbS5hcG9zdGlsbGUucm9vdCI6IiRkaXNhYmxlZCJ9fQ.ggV3fetHHKAme7s_4l2wMBhcBigRGJG12HVVbE9Dw_pwxWzlbgMaLSk8PELONxAz5cdPyt5cQNf5DrW_-4aBpaISFnyZMjwc7lTndZOjK1iPw2jgOX9JuTvtWMEVh6iXPRm1SZeY2j9UIhjGYdF-FOsy9uazuS0-8DyJF9LuPlWQychfiHgsEkXpBPoxpJ5VuuRDAuxxgJrYlTsNMS05CvcR18t_fx9FxMEE2VCtjMwRswlBGgznwVXQo0n1YMxW3otNDWvnYJkbGjtTs_PE-5GqdLJCIYw8GcavjWwto6RDNO8_dakeUE7t68fRxmRhGFa1YDSKyIAPQg6wyANkiQ gunicorn-registry stdout | 2022-05-05 10:15:13,528 [224] [DEBUG] [auth.permissions] Identity loaded: gunicorn-registry stdout | 2022-05-05 10:15:13,528 [224] [DEBUG] [auth.permissions] Loaded signed_jwt identity for: ibazulic gunicorn-registry stdout | 2022-05-05 10:15:13,528 [224] [DEBUG] [auth.registry_jwt_auth] Identity changed to ibazulic gunicorn-registry stdout | 2022-05-05 10:15:13,529 [224] [DEBUG] [auth.registry_jwt_auth] Auth context set to {'version': 2, 'entity_kind': 'user', 'entity_reference': '105a42ac-0110-4aed-8579-1c532997244a', 'kind': 'user', 'user': 'ibazulic', 'com.apostille.roots': {'ibazulic/test-big-layers': '$disabled'}, 'com.apostille.root': '$disabled'} gunicorn-registry stdout | 2022-05-05 10:15:13,529 [224] [DEBUG] [endpoints.v2] Checking permission for repo: ibazulic/test-big-layers gunicorn-registry stdout | 2022-05-05 10:15:13,530 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_user_id", "t1"."name", "t1"."visibility_id", "t1"."description", "t1"."badge_token", "t1"."kind_id", "t1"."trust_enabled", "t1"."state", "t2"."id", "t2"."uuid", "t2"."username", "t2"."password_hash", "t2"."email", "t2"."verified", "t2"."stripe_id", "t2"."organization", "t2"."robot", "t2"."invoice_email", "t2"."invalid_login_attempts", "t2"."last_invalid_login", "t2"."removed_tag_expiration_s", "t2"."enabled", "t2"."invoice_email_address", "t2"."given_name", "t2"."family_name", "t2"."company", "t2"."location", "t2"."maximum_queued_builds_count", "t2"."creation_date", "t2"."last_accessed" FROM "repository" AS "t1" INNER JOIN "user" AS "t2" ON ("t1"."namespace_user_id" = "t2"."id") WHERE ((("t2"."username" = %s) AND ("t1"."name" = %s)) AND ("t1"."state" != %s)) LIMIT %s OFFSET %s', ['ibazulic', 'test-big-layers', 3, 1, 0]) gunicorn-registry stdout | 2022-05-05 10:15:13,533 [224] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."uuid", "t1"."byte_count", "t1"."sha_state", "t1"."location_id", "t1"."storage_metadata", "t1"."chunk_count", "t1"."uncompressed_byte_count", "t1"."created", "t1"."piece_sha_state", "t1"."piece_hashes", "t2"."id", "t2"."name" FROM "blobupload" AS "t1" INNER JOIN "imagestoragelocation" AS "t2" ON ("t1"."location_id" = "t2"."id") WHERE ("t1"."uuid" = %s) LIMIT %s OFFSET %s', ['8c27a1e5-8725-40a2-a184-f4e11fe5bc86', 1, 0]) gunicorn-registry stdout | 2022-05-05 10:15:13,535 [224] [DEBUG] [data.database] Disconnecting from database. gunicorn-registry stdout | 2022-05-05 10:15:13,535 [224] [DEBUG] [peewee.pool] Returning 140384019510832 to pool. namespacegcworker stdout | 2022-05-05 10:15:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:15:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:38.977973+00:00 (in 24.690995 seconds) namespacegcworker stdout | 2022-05-05 10:15:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:14 UTC)" (scheduled at 2022-05-05 10:15:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:15:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:15:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:15:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:15:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:44.479974+00:00 (in 29.999482 seconds) buildlogsarchiver stdout | 2022-05-05 10:15:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:44 UTC)" (scheduled at 2022-05-05 10:15:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:15:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 15, 14, 480902), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:15:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:15:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:15:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:15:17,369 [219] [DEBUG] [app] Starting request: urn:request:cd3cc0a6-66c2-40f8-815d-dcc7b2efa67d (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:17,369 [221] [DEBUG] [app] Starting request: urn:request:d15757a3-abee-4547-b95c-121445e3f77b (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:17,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:17,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:15:17,374 [231] [DEBUG] [app] Starting request: urn:request:1ca76045-30cd-4a9d-94fc-1b8a088b5586 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:17,374 [228] [DEBUG] [app] Starting request: urn:request:3fa7a5e2-5fe2-4296-b0b7-3cb2776ccbcd (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:17,374 [228] [DEBUG] [app] Ending request: urn:request:3fa7a5e2-5fe2-4296-b0b7-3cb2776ccbcd (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:17,374 [231] [DEBUG] [app] Ending request: urn:request:1ca76045-30cd-4a9d-94fc-1b8a088b5586 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:15:17,375 [231] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:15:17,375 [228] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:15:17,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:17,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:17,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:17,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:17,378 [221] [DEBUG] [app] Starting request: urn:request:9dbc2047-ccef-4b0d-bd3b-2dfabbb0677c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:17,379 [222] [DEBUG] [app] Starting request: urn:request:64dcfc0b-4840-4132-a13f-80cf5c2505ca (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:17,379 [221] [DEBUG] [app] Ending request: urn:request:9dbc2047-ccef-4b0d-bd3b-2dfabbb0677c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:17,379 [222] [DEBUG] [app] Ending request: urn:request:64dcfc0b-4840-4132-a13f-80cf5c2505ca (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:17,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:15:17,380 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:15:17,380 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:17,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:15:17,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:15:17,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:15:17,388 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:15:17,389 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:15:17,395 [221] [DEBUG] [app] Ending request: urn:request:d15757a3-abee-4547-b95c-121445e3f77b (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:17,395 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:15:17,396 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:15:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:15:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.027) gunicorn-web stdout | 2022-05-05 10:15:17,397 [219] [DEBUG] [app] Ending request: urn:request:cd3cc0a6-66c2-40f8-815d-dcc7b2efa67d (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:17,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:15:17,397 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:15:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:15:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.028) gunicorn-registry stdout | 2022-05-05 10:15:20,987 [231] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '231'} gunicorn-registry stdout | 2022-05-05 10:15:20,993 [230] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '230'} gunicorn-registry stdout | 2022-05-05 10:15:20,997 [226] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '226'} gunicorn-registry stdout | 2022-05-05 10:15:21,017 [229] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '229'} gunicorn-registry stdout | 2022-05-05 10:15:21,022 [228] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '228'} gunicorn-registry stdout | 2022-05-05 10:15:21,025 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:15:21,037 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} gunicorn-registry stdout | 2022-05-05 10:15:21,040 [227] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '227'} gunicorn-registry stdout | 2022-05-05 10:15:22,893 [224] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '224'} notificationworker stdout | 2022-05-05 10:15:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:15:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:32.207092+00:00 (in 9.116622 seconds) notificationworker stdout | 2022-05-05 10:15:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:33 UTC)" (scheduled at 2022-05-05 10:15:23.089985+00:00) notificationworker stdout | 2022-05-05 10:15:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:15:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 23, 90980), True, datetime.datetime(2022, 5, 5, 10, 15, 23, 90980), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:15:23,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:15:23,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:15:23,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:15:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:15:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:13.394619+00:00 (in 48.409156 seconds) repositorygcworker stdout | 2022-05-05 10:15:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:24 UTC)" (scheduled at 2022-05-05 10:15:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:15:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:15:24,986 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 24, 985941), True, datetime.datetime(2022, 5, 5, 10, 15, 24, 985941), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:15:24,997 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:15:24,997 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:15:24,997 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:15:25,720 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:15:26,313 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:15:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:15:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:56.865082+00:00 (in 29.999515 seconds) gcworker stdout | 2022-05-05 10:15:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:56 UTC)" (scheduled at 2022-05-05 10:15:26.865082+00:00) gcworker stdout | 2022-05-05 10:15:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536126866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:15:26,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:15:26,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:15:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:15:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:15:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:58.201650+00:00 (in 29.999510 seconds) securityworker stdout | 2022-05-05 10:15:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:58 UTC)" (scheduled at 2022-05-05 10:15:28.201650+00:00) securityworker stdout | 2022-05-05 10:15:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:15:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:15:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:15:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:15:28,216 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:15:28,218 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:15:28,220 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:15:28,220 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:15:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 10, 28, 220976), 1, 2]) securityworker stdout | 2022-05-05 10:15:28,223 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:15:28,223 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:15:28,224 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 10, 28, 224107), 1, 2]) securityworker stdout | 2022-05-05 10:15:28,233 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:15:28,233 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:15:28,233 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:15:28,233 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:15:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:15:30,090 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:15:30,267 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:15:30,613 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:15:31,526 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:15:31,557 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} securityscanningnotificationworker stdout | 2022-05-05 10:15:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:15:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:35.969483+00:00 (in 4.194984 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:15:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:31 UTC)" (scheduled at 2022-05-05 10:15:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:15:31,774 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:15:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 31, 775020), True, datetime.datetime(2022, 5, 5, 10, 15, 31, 775020), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:15:31,785 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:15:31,785 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:15:31,785 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:31 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:15:31,851 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} notificationworker stdout | 2022-05-05 10:15:32,207 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:15:32,207 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:33.089985+00:00 (in 0.882377 seconds) notificationworker stdout | 2022-05-05 10:15:32,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:20:32 UTC)" (scheduled at 2022-05-05 10:15:32.207092+00:00) notificationworker stdout | 2022-05-05 10:15:32,208 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 15, 32, 208174), 'notification/%']) notificationworker stdout | 2022-05-05 10:15:32,218 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 15, 32, 208174), True, datetime.datetime(2022, 5, 5, 10, 15, 32, 208174), 0, 'notification/%']) notificationworker stdout | 2022-05-05 10:15:32,220 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 15, 32, 208174), True, datetime.datetime(2022, 5, 5, 10, 15, 32, 208174), 0, 'notification/%', False, datetime.datetime(2022, 5, 5, 10, 15, 32, 208174), 'notification/%']) notificationworker stdout | 2022-05-05 10:15:32,222 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:15:32,222 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:20:32 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:15:32,369 [221] [DEBUG] [app] Starting request: urn:request:3fd8a384-e8ed-4373-a751-a95eb82eab0a (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:32,369 [219] [DEBUG] [app] Starting request: urn:request:b8aa1a18-09f2-4788-ac3c-1a8a60be0cea (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:32,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:32,373 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:15:32,374 [224] [DEBUG] [app] Starting request: urn:request:04ed89cf-fbf0-4f9e-8abd-18c2ec7196a2 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:32,374 [230] [DEBUG] [app] Starting request: urn:request:0d2c0b59-a2cd-46e8-b4ff-4f53c3e638f2 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:32,375 [224] [DEBUG] [app] Ending request: urn:request:04ed89cf-fbf0-4f9e-8abd-18c2ec7196a2 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:32,375 [230] [DEBUG] [app] Ending request: urn:request:0d2c0b59-a2cd-46e8-b4ff-4f53c3e638f2 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:15:32,375 [224] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:15:32,376 [230] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:15:32,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:15:32,376 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:32,378 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:32,379 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:32,380 [220] [DEBUG] [app] Starting request: urn:request:ceabfac1-28ee-4a39-94d6-cac916bf495f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:32,380 [219] [DEBUG] [app] Starting request: urn:request:432d0df2-070d-472e-9f5c-2b6b12ad76ff (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:32,380 [220] [DEBUG] [app] Ending request: urn:request:ceabfac1-28ee-4a39-94d6-cac916bf495f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:32,381 [219] [DEBUG] [app] Ending request: urn:request:432d0df2-070d-472e-9f5c-2b6b12ad76ff (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:15:32,381 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:15:32,381 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.003) gunicorn-web stdout | 2022-05-05 10:15:32,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:32,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:15:32,383 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:32,384 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:15:32,389 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:15:32,391 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:15:32,396 [219] [DEBUG] [app] Ending request: urn:request:b8aa1a18-09f2-4788-ac3c-1a8a60be0cea (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:32,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:15:32,397 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:15:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:15:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.028) gunicorn-web stdout | 2022-05-05 10:15:32,398 [221] [DEBUG] [app] Ending request: urn:request:3fd8a384-e8ed-4373-a751-a95eb82eab0a (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:32,398 [221] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:15:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.031 118 0.030) gunicorn-web stdout | 2022-05-05 10:15:32,399 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:15:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-secscan stdout | 2022-05-05 10:15:33,064 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:15:33,068 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:15:33,072 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:15:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:15:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:43.089985+00:00 (in 9.999429 seconds) notificationworker stdout | 2022-05-05 10:15:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:43 UTC)" (scheduled at 2022-05-05 10:15:33.089985+00:00) notificationworker stdout | 2022-05-05 10:15:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:15:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 33, 91066), True, datetime.datetime(2022, 5, 5, 10, 15, 33, 91066), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:15:33,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:15:33,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:15:33,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:43 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-05 10:15:33,231 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:15:33,329 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:15:33,711 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:15:33,851 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:15:34,000 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:15:34,734 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:15:35,743 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:15:35,918 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} securityscanningnotificationworker stdout | 2022-05-05 10:15:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:15:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:31.774027+00:00 (in 55.803903 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:15:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:35 UTC)" (scheduled at 2022-05-05 10:15:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:15:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:15:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:35 UTC)" executed successfully globalpromstats stdout | 2022-05-05 10:15:35,975 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:15:36,074 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:15:36,088 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:15:36,101 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:15:36,108 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:15:36,111 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:15:36,187 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:15:36,504 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:15:38,153 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:15:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:15:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:14.286471+00:00 (in 35.308028 seconds) namespacegcworker stdout | 2022-05-05 10:15:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:38 UTC)" (scheduled at 2022-05-05 10:15:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:15:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:15:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 38, 978975), True, datetime.datetime(2022, 5, 5, 10, 15, 38, 978975), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:15:38,990 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:15:38,990 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:15:38,991 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:15:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:15:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:53.089985+00:00 (in 9.999503 seconds) notificationworker stdout | 2022-05-05 10:15:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:53 UTC)" (scheduled at 2022-05-05 10:15:43.089985+00:00) notificationworker stdout | 2022-05-05 10:15:43,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:15:43,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 43, 90910), True, datetime.datetime(2022, 5, 5, 10, 15, 43, 90910), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:15:43,432 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:15:43,432 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:15:43,432 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:15:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:15:44,508 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:15:44,509 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:14.479974+00:00 (in 29.971054 seconds) buildlogsarchiver stdout | 2022-05-05 10:15:44,509 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:14 UTC)" (scheduled at 2022-05-05 10:15:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:15:44,510 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 15, 44, 509298), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:15:44,523 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:15:44,523 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:15:44,523 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:15:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:15:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:15:59.184038+00:00 (in 11.996015 seconds) exportactionlogsworker stdout | 2022-05-05 10:15:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:47 UTC)" (scheduled at 2022-05-05 10:15:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:15:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:15:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:16:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:15:47,370 [222] [DEBUG] [app] Starting request: urn:request:d65f646b-ba42-4d2f-9bf8-92971c331ed1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:47,370 [220] [DEBUG] [app] Starting request: urn:request:f94ece9c-cd08-402a-98ba-0c94af9d9c2d (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:47,371 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 2022/05/05 10:15:47 [error] 106#0: *769 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 127.0.0.1, server: , request: "GET /v1/_internal_ping HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_registry.sock:/v1/_internal_ping", host: "localhost:8080" gunicorn-web stdout | 2022-05-05 10:15:47,801 [222] [DEBUG] [app] Starting request: urn:request:8063144d-eb8a-475c-abc2-503109b35494 (/quay-registry/static/502.html) gunicorn-web stdout | 2022-05-05 10:15:47,803 [222] [DEBUG] [app] Ending request: urn:request:8063144d-eb8a-475c-abc2-503109b35494 (/quay-registry/static/502.html) gunicorn-web stdout | 2022-05-05 10:15:47,803 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:47 +0000] "GET /quay-registry/static/502.html HTTP/1.0" 308 301 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 502 301 "-" "python-requests/2.22.0" (0.431 162 0.459 : 0.004) gunicorn-web stdout | 2022-05-05 10:15:47,804 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 502 301 gunicorn-web stdout | 2022-05-05 10:15:48,236 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:48,673 [220] [DEBUG] [app] Starting request: urn:request:6387855c-492f-4548-baed-f44a63d4b4d4 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:48,674 [220] [DEBUG] [app] Ending request: urn:request:6387855c-492f-4548-baed-f44a63d4b4d4 (/_internal_ping) nginx stdout | 2022/05/05 10:15:48 [error] 106#0: *770 upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1, server: , request: "GET /v1/_internal_ping HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_registry.sock:/v1/_internal_ping", host: "localhost:8080" gunicorn-web stdout | 2022-05-05 10:15:48,679 [219] [DEBUG] [app] Starting request: urn:request:d8b6c8c0-3135-40bb-a8ae-bca1e725270e (/quay-registry/static/502.html) gunicorn-web stdout | 2022-05-05 10:15:48,681 [219] [DEBUG] [app] Ending request: urn:request:d8b6c8c0-3135-40bb-a8ae-bca1e725270e (/quay-registry/static/502.html) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:48 +0000] "GET /v1/_internal_ping HTTP/1.1" 502 301 "-" "python-requests/2.22.0" (1.309 162 1.338 : 0.002) gunicorn-web stdout | 2022-05-05 10:15:48,681 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:48 +0000] "GET /quay-registry/static/502.html HTTP/1.0" 308 301 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:15:48,681 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 502 301 gunicorn-web stdout | 2022-05-05 10:15:48,684 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:15:48,686 [219] [DEBUG] [app] Starting request: urn:request:e42c075c-1648-4797-8f65-4ce1c06016b6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:48,687 [219] [DEBUG] [app] Ending request: urn:request:e42c075c-1648-4797-8f65-4ce1c06016b6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:15:48,687 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:48 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:48 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:15:48,687 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:48,688 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:15:48,781 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:15:48,788 [222] [WARNING] [health.healthcheck] [FAILED HEALTH CHECK] {'services_expanded': {'registry_gunicorn': {'status': False, 'failure': 'Got non-200 response for worker: 502'}, 'web_gunicorn': {'status': True}, 'service_key': {'status': True}, 'disk_space': {'status': True}, 'database': {'status': True}, 'auth': {'status': True}}, 'notes': [], 'is_testing': False, 'config_provider': 'k8s', 'local_service_key_id': 'I-4Izl8kZruZ4ivvJ-1E-BaYSwe6d_qXDEmZvFOJIo0', 'hostname': 'quay-quay-app-d6556d95c-xg4lc'} gunicorn-web stdout | 2022-05-05 10:15:48,788 [222] [DEBUG] [app] Ending request: urn:request:d65f646b-ba42-4d2f-9bf8-92971c331ed1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:48,789 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:15:48,789 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:15:48 +0000] "GET /health/instance HTTP/1.0" 503 153 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:15:48 +0000] "GET /health/instance HTTP/1.1" 503 153 "-" "kube-probe/1.22" (1.420 118 1.448) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:15:49 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.430 159 0.430) gunicorn-web stdout | 2022-05-05 10:15:49,103 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:15:49 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:15:49,103 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:15:49,104 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:15:49,556 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:15:49,564 [220] [WARNING] [health.healthcheck] [FAILED HEALTH CHECK] {'services_expanded': {'registry_gunicorn': {'status': False, 'failure': 'Got non-200 response for worker: 502'}, 'web_gunicorn': {'status': True}, 'service_key': {'status': True}, 'disk_space': {'status': True}, 'database': {'status': True}, 'auth': {'status': True}}, 'notes': [], 'is_testing': False, 'config_provider': 'k8s', 'local_service_key_id': 'I-4Izl8kZruZ4ivvJ-1E-BaYSwe6d_qXDEmZvFOJIo0', 'hostname': 'quay-quay-app-d6556d95c-xg4lc'} gunicorn-web stdout | 2022-05-05 10:15:49,564 [220] [DEBUG] [app] Ending request: urn:request:f94ece9c-cd08-402a-98ba-0c94af9d9c2d (/health/instance) gunicorn-web stdout | 2022-05-05 10:15:49,565 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:15:49,565 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:15:49 +0000] "GET /health/instance HTTP/1.0" 503 153 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:15:49 +0000] "GET /health/instance HTTP/1.1" 503 153 "-" "kube-probe/1.22" (2.196 118 2.224) gunicorn-registry stdout | 2022-05-05 10:15:51,287 [225] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '225'} gunicorn-registry stdout | 2022-05-05 10:15:51,301 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} nginx stdout | 2022/05/05 10:15:51 [error] 106#0: *473 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.128.2.6, server: , request: "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/a1836993-a72f-466e-acde-71c412576855 HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_registry.sock:/v2/ibazulic/test-big-layers/blobs/uploads/a1836993-a72f-466e-acde-71c412576855", host: "quay-quay-quay-enterprise.apps.rcegan.azemea.com" nginx stdout | 2022/05/05 10:15:51 [error] 106#0: *473 client intended to send too large body: 2516758573 bytes, client: 10.128.2.6, server: , request: "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/a1836993-a72f-466e-acde-71c412576855 HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_registry.sock/v2/ibazulic/test-big-layers/blobs/uploads/a1836993-a72f-466e-acde-71c412576855", host: "quay-quay-quay-enterprise.apps.rcegan.azemea.com" nginx stdout | 2022/05/05 10:15:51 [error] 106#0: *739 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 10.128.2.6, server: , request: "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/8c27a1e5-8725-40a2-a184-f4e11fe5bc86 HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_registry.sock:/v2/ibazulic/test-big-layers/blobs/uploads/8c27a1e5-8725-40a2-a184-f4e11fe5bc86", host: "quay-quay-quay-enterprise.apps.rcegan.azemea.com" nginx stdout | 2022/05/05 10:15:51 [error] 106#0: *739 client intended to send too large body: 92949571 bytes, client: 10.128.2.6, server: , request: "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/8c27a1e5-8725-40a2-a184-f4e11fe5bc86 HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_registry.sock/v2/ibazulic/test-big-layers/blobs/uploads/8c27a1e5-8725-40a2-a184-f4e11fe5bc86", host: "quay-quay-quay-enterprise.apps.rcegan.azemea.com" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:15:51 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/8c27a1e5-8725-40a2-a184-f4e11fe5bc86 HTTP/1.1" 413 199 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (38.112 92997279 37.821) nginx stdout | 10.128.2.6 () - - [05/May/2022:10:15:51 +0000] "PATCH /v2/ibazulic/test-big-layers/blobs/uploads/a1836993-a72f-466e-acde-71c412576855 HTTP/1.1" 413 199 "-" "docker/20.10.14 go/go1.16.15 git-commit/87a90dc kernel/5.10.0-13-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.14 \x5C(linux\x5C))" (339.307 2518083231 338.989) notificationworker stdout | 2022-05-05 10:15:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:15:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:03.089985+00:00 (in 9.999284 seconds) notificationworker stdout | 2022-05-05 10:15:53,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:03 UTC)" (scheduled at 2022-05-05 10:15:53.089985+00:00) notificationworker stdout | 2022-05-05 10:15:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:15:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 15, 53, 91176), True, datetime.datetime(2022, 5, 5, 10, 15, 53, 91176), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:15:53,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:15:53,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:15:53,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:15:55,758 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:15:56,343 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:15:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:15:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:26.865082+00:00 (in 29.999485 seconds) gcworker stdout | 2022-05-05 10:15:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:26 UTC)" (scheduled at 2022-05-05 10:15:56.865082+00:00) gcworker stdout | 2022-05-05 10:15:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536156866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:15:56,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:15:56,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:15:56,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:15:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:15:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:28.201650+00:00 (in 29.999513 seconds) securityworker stdout | 2022-05-05 10:15:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:28 UTC)" (scheduled at 2022-05-05 10:15:58.201650+00:00) securityworker stdout | 2022-05-05 10:15:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:15:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:15:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:15:58,208 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:15:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:15:58,216 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:28 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:15:59,184 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:15:59,184 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:00.978909+00:00 (in 1.794337 seconds) exportactionlogsworker stdout | 2022-05-05 10:15:59,184 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:20:59 UTC)" (scheduled at 2022-05-05 10:15:59.184038+00:00) exportactionlogsworker stdout | 2022-05-05 10:15:59,185 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 15, 59, 185018), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:15:59,195 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 15, 59, 185018), True, datetime.datetime(2022, 5, 5, 10, 15, 59, 185018), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:15:59,197 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 15, 59, 185018), True, datetime.datetime(2022, 5, 5, 10, 15, 59, 185018), 0, 'exportactionlogs/%', False, datetime.datetime(2022, 5, 5, 10, 15, 59, 185018), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:15:59,199 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:15:59,199 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:20:59 UTC)" executed successfully gcworker stdout | 2022-05-05 10:16:00,121 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:16:00,304 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:16:00,651 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:16:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:16:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:47.187465+00:00 (in 46.208076 seconds) exportactionlogsworker stdout | 2022-05-05 10:16:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:00 UTC)" (scheduled at 2022-05-05 10:16:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:16:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:16:00,980 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 0, 979854), True, datetime.datetime(2022, 5, 5, 10, 16, 0, 979854), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:16:00,990 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:16:00,990 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:16:00,990 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:00 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:16:01,554 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:16:01,567 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:16:01,888 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:16:02,369 [222] [DEBUG] [app] Starting request: urn:request:3249140b-c4d7-4fe3-93ce-8843d89bbf9c (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:02,369 [221] [DEBUG] [app] Starting request: urn:request:41be079a-727b-4c41-b651-6b47900d0045 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:02,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:02,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:16:02,375 [288] [DEBUG] [app] Starting request: urn:request:f2aa0bbd-4310-46cb-93ff-fdccfdfbc11c (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:02,376 [292] [DEBUG] [app] Starting request: urn:request:3ca3db07-6b7f-47e4-b436-7720e9f26190 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:02,378 [288] [DEBUG] [app] Ending request: urn:request:f2aa0bbd-4310-46cb-93ff-fdccfdfbc11c (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:02,378 [292] [DEBUG] [app] Ending request: urn:request:3ca3db07-6b7f-47e4-b436-7720e9f26190 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.007) gunicorn-registry stdout | 2022-05-05 10:16:02,379 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:02,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.007) gunicorn-registry stdout | 2022-05-05 10:16:02,379 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:02,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:02,381 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:02,382 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:02,383 [219] [DEBUG] [app] Starting request: urn:request:5f4d018d-8d92-41c9-ba7c-ed533c77c4e1 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:02,383 [219] [DEBUG] [app] Ending request: urn:request:5f4d018d-8d92-41c9-ba7c-ed533c77c4e1 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:02,383 [221] [DEBUG] [app] Starting request: urn:request:4170f8aa-2156-4d50-acbe-2fde1bcd3fb4 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:02,384 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:16:02,384 [221] [DEBUG] [app] Ending request: urn:request:4170f8aa-2156-4d50-acbe-2fde1bcd3fb4 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:02,384 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:02,385 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:16:02,385 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:02,385 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:16:02,385 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:16:02,392 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:16:02,392 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:16:02,398 [221] [DEBUG] [app] Ending request: urn:request:41be079a-727b-4c41-b651-6b47900d0045 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:02,398 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:16:02,398 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:16:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:16:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) gunicorn-web stdout | 2022-05-05 10:16:02,399 [222] [DEBUG] [app] Ending request: urn:request:3249140b-c4d7-4fe3-93ce-8843d89bbf9c (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:02,399 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:16:02,400 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:16:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:16:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.031) gunicorn-secscan stdout | 2022-05-05 10:16:03,072 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:16:03,076 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:16:03,080 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:16:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:16:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:12.207990+00:00 (in 9.117552 seconds) notificationworker stdout | 2022-05-05 10:16:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:13 UTC)" (scheduled at 2022-05-05 10:16:03.089985+00:00) notificationworker stdout | 2022-05-05 10:16:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:16:03,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 3, 90898), True, datetime.datetime(2022, 5, 5, 10, 16, 3, 90898), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:16:03,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:16:03,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:16:03,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:13 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-05 10:16:03,264 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:16:03,362 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:16:03,743 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:16:03,859 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:16:04,012 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:16:04,761 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:16:05,778 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:16:05,956 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:16:06,010 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:16:06,083 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:16:06,096 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:16:06,109 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:16:06,115 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:16:06,119 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:16:06,223 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:16:06,539 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:16:08,163 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:16:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:16:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:13.089985+00:00 (in 0.881531 seconds) notificationworker stdout | 2022-05-05 10:16:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:12 UTC)" (scheduled at 2022-05-05 10:16:12.207990+00:00) notificationworker stdout | 2022-05-05 10:16:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:16:12,208 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:16:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:16:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:23.089985+00:00 (in 9.999461 seconds) notificationworker stdout | 2022-05-05 10:16:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:23 UTC)" (scheduled at 2022-05-05 10:16:13.089985+00:00) notificationworker stdout | 2022-05-05 10:16:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:16:13,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 13, 91010), True, datetime.datetime(2022, 5, 5, 10, 16, 13, 91010), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:16:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:16:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:16:13,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:16:13,394 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:16:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:24.984990+00:00 (in 11.589908 seconds) repositorygcworker stdout | 2022-05-05 10:16:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:13 UTC)" (scheduled at 2022-05-05 10:16:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:16:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:16:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:16:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:16:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:38.977973+00:00 (in 24.691011 seconds) namespacegcworker stdout | 2022-05-05 10:16:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:14 UTC)" (scheduled at 2022-05-05 10:16:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:16:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:16:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:16:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:16:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:44.479974+00:00 (in 29.999484 seconds) buildlogsarchiver stdout | 2022-05-05 10:16:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:44 UTC)" (scheduled at 2022-05-05 10:16:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:16:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 16, 14, 480953), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:16:14,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:16:14,492 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:16:14,493 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:16:17,369 [222] [DEBUG] [app] Starting request: urn:request:881bd57a-a3cc-4ab8-b04c-e99fb9801fb4 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:17,369 [221] [DEBUG] [app] Starting request: urn:request:48c16e7a-4be8-4ed3-aaf2-367a98a15f10 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:17,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:16:17,372 [292] [DEBUG] [app] Starting request: urn:request:5fbb4cf8-2f31-470d-ac8c-87c3efa88c08 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:17,373 [292] [DEBUG] [app] Ending request: urn:request:5fbb4cf8-2f31-470d-ac8c-87c3efa88c08 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:17,373 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:16:17,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:17,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:16:17,375 [290] [DEBUG] [app] Starting request: urn:request:4e22fa37-d4ad-4a08-b6fa-a792b6e2dccc (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:17,377 [222] [DEBUG] [app] Starting request: urn:request:2d613bce-dd7a-431e-8967-de85541952b7 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:17,378 [222] [DEBUG] [app] Ending request: urn:request:2d613bce-dd7a-431e-8967-de85541952b7 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:17,378 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-registry stdout | 2022-05-05 10:16:17,378 [290] [DEBUG] [app] Ending request: urn:request:4e22fa37-d4ad-4a08-b6fa-a792b6e2dccc (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:17,378 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:17,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:16:17,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.007 162 0.007) gunicorn-registry stdout | 2022-05-05 10:16:17,379 [290] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:17,381 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:17,383 [221] [DEBUG] [app] Starting request: urn:request:09594b5b-7709-4dab-8bbd-fddc754efc2f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:17,383 [221] [DEBUG] [app] Ending request: urn:request:09594b5b-7709-4dab-8bbd-fddc754efc2f (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:16:17,384 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:17,385 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:17,385 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:16:17,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:16:17,390 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:16:17,393 [222] [DEBUG] [app] Ending request: urn:request:881bd57a-a3cc-4ab8-b04c-e99fb9801fb4 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:17,393 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:16:17,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:16:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:16:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:16:17,396 [221] [DEBUG] [app] Ending request: urn:request:48c16e7a-4be8-4ed3-aaf2-367a98a15f10 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:17,396 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:16:17,397 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:16:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:16:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) gunicorn-registry stdout | 2022-05-05 10:16:21,315 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:21,320 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:21,324 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:21,327 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:21,331 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:21,334 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:21,338 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:21,342 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:21,345 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:16:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:16:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:33.089985+00:00 (in 9.999509 seconds) notificationworker stdout | 2022-05-05 10:16:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:33 UTC)" (scheduled at 2022-05-05 10:16:23.089985+00:00) notificationworker stdout | 2022-05-05 10:16:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:16:23,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 23, 90967), True, datetime.datetime(2022, 5, 5, 10, 16, 23, 90967), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:16:23,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:16:23,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:16:23,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:16:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:16:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:13.394619+00:00 (in 48.409165 seconds) repositorygcworker stdout | 2022-05-05 10:16:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:24 UTC)" (scheduled at 2022-05-05 10:16:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:16:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:16:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 24, 985925), True, datetime.datetime(2022, 5, 5, 10, 16, 24, 985925), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:16:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:16:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:16:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:16:25,795 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:16:26,354 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:16:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:16:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:56.865082+00:00 (in 29.999487 seconds) gcworker stdout | 2022-05-05 10:16:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:56 UTC)" (scheduled at 2022-05-05 10:16:26.865082+00:00) gcworker stdout | 2022-05-05 10:16:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536186866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:16:26,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:16:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:16:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:16:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:16:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:58.201650+00:00 (in 29.999530 seconds) securityworker stdout | 2022-05-05 10:16:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:58 UTC)" (scheduled at 2022-05-05 10:16:28.201650+00:00) securityworker stdout | 2022-05-05 10:16:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:16:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:16:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:16:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:16:28,215 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:16:28,217 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:16:28,219 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:16:28,219 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:16:28,220 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 11, 28, 219443), 1, 2]) securityworker stdout | 2022-05-05 10:16:28,221 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:16:28,221 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:16:28,222 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 11, 28, 221685), 1, 2]) securityworker stdout | 2022-05-05 10:16:28,224 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:16:28,224 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:16:28,224 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:16:28,224 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:16:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:16:30,129 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:16:30,341 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:16:30,688 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} repositorygcworker stdout | 2022-05-05 10:16:31,591 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:16:31,598 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} securityscanningnotificationworker stdout | 2022-05-05 10:16:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:16:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:35.969483+00:00 (in 4.194801 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:16:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:31 UTC)" (scheduled at 2022-05-05 10:16:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:16:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:16:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 31, 775168), True, datetime.datetime(2022, 5, 5, 10, 16, 31, 775168), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:16:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:16:31,786 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:16:31,786 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:31 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:16:31,925 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:16:32,370 [220] [DEBUG] [app] Starting request: urn:request:dba86548-692f-4a71-a382-bd6ad060ac0f (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:32,370 [219] [DEBUG] [app] Starting request: urn:request:cb288d44-0791-4576-b296-cfd1987a14f0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:32,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:32,371 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:16:32,373 [290] [DEBUG] [app] Starting request: urn:request:1c86f80c-b47e-4aa0-be5c-83815978e93f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:32,373 [292] [DEBUG] [app] Starting request: urn:request:aa4e54c8-aa60-42da-bc73-82ea0ac804e8 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:32,374 [290] [DEBUG] [app] Ending request: urn:request:1c86f80c-b47e-4aa0-be5c-83815978e93f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:32,374 [292] [DEBUG] [app] Ending request: urn:request:aa4e54c8-aa60-42da-bc73-82ea0ac804e8 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:32,374 [290] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:16:32,374 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:32,374 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:32,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:16:32,377 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:32,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:32,378 [220] [DEBUG] [app] Starting request: urn:request:fe4f2ccf-1d8d-4a8d-b701-a96a6aa69a23 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:32,378 [221] [DEBUG] [app] Starting request: urn:request:b38c23ca-6eaf-4868-84ec-ce6d28f077a8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:32,378 [220] [DEBUG] [app] Ending request: urn:request:fe4f2ccf-1d8d-4a8d-b701-a96a6aa69a23 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:32,379 [221] [DEBUG] [app] Ending request: urn:request:b38c23ca-6eaf-4868-84ec-ce6d28f077a8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:32,379 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:16:32,379 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:32,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:32,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:32,380 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:16:32,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:16:32,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:16:32,388 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:16:32,394 [219] [DEBUG] [app] Ending request: urn:request:cb288d44-0791-4576-b296-cfd1987a14f0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:32,394 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:16:32,394 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:16:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:16:32,394 [220] [DEBUG] [app] Ending request: urn:request:dba86548-692f-4a71-a382-bd6ad060ac0f (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:32,395 [220] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:16:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:16:32,395 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:16:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:16:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-secscan stdout | 2022-05-05 10:16:33,081 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:16:33,087 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} notificationworker stdout | 2022-05-05 10:16:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:16:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:43.089985+00:00 (in 9.999499 seconds) gunicorn-secscan stdout | 2022-05-05 10:16:33,090 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:16:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:43 UTC)" (scheduled at 2022-05-05 10:16:33.089985+00:00) notificationworker stdout | 2022-05-05 10:16:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:16:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 33, 91094), True, datetime.datetime(2022, 5, 5, 10, 16, 33, 91094), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:16:33,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:16:33,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:16:33,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:43 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-05 10:16:33,293 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:16:33,374 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:16:33,780 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:16:33,896 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:16:34,050 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:16:34,778 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:16:35,794 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} securityscanningnotificationworker stdout | 2022-05-05 10:16:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:16:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:31.774027+00:00 (in 55.804052 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:16:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:35 UTC)" (scheduled at 2022-05-05 10:16:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:16:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:16:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:35 UTC)" executed successfully servicekey stdout | 2022-05-05 10:16:35,978 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:16:36,037 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:16:36,091 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:16:36,104 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:16:36,117 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:16:36,125 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:16:36,128 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:16:36,253 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:16:36,568 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:16:38,200 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:16:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:16:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:14.286471+00:00 (in 35.307939 seconds) namespacegcworker stdout | 2022-05-05 10:16:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:38 UTC)" (scheduled at 2022-05-05 10:16:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:16:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:16:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 38, 979097), True, datetime.datetime(2022, 5, 5, 10, 16, 38, 979097), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:16:38,990 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:16:38,990 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:16:38,990 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:16:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:16:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:16:53.089985+00:00 (in 9.999429 seconds) notificationworker stdout | 2022-05-05 10:16:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:53 UTC)" (scheduled at 2022-05-05 10:16:43.089985+00:00) notificationworker stdout | 2022-05-05 10:16:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:16:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 43, 91077), True, datetime.datetime(2022, 5, 5, 10, 16, 43, 91077), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:16:43,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:16:43,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:16:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:16:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:16:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:16:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:14.479974+00:00 (in 29.999479 seconds) buildlogsarchiver stdout | 2022-05-05 10:16:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:14 UTC)" (scheduled at 2022-05-05 10:16:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:16:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 16, 44, 480945), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:16:44,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:16:44,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:16:44,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:16:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:16:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:00.978909+00:00 (in 13.790983 seconds) exportactionlogsworker stdout | 2022-05-05 10:16:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:47 UTC)" (scheduled at 2022-05-05 10:16:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:16:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:16:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:17:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:16:47,369 [221] [DEBUG] [app] Starting request: urn:request:8dbb4615-6dde-42bf-8d9a-e59974d585a1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:47,369 [222] [DEBUG] [app] Starting request: urn:request:dbf750a1-36e4-47d7-a622-bb82a3f52ead (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:47,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:16:47,372 [288] [DEBUG] [app] Starting request: urn:request:dbd783fa-af39-40fe-a14c-79794ccc4531 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:47,373 [290] [DEBUG] [app] Starting request: urn:request:c8e85bf3-c3ee-4d17-a260-ed7c9f320496 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:47,373 [288] [DEBUG] [app] Ending request: urn:request:dbd783fa-af39-40fe-a14c-79794ccc4531 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:47,373 [290] [DEBUG] [app] Ending request: urn:request:c8e85bf3-c3ee-4d17-a260-ed7c9f320496 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:16:47,373 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:16:47,374 [290] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:47,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:47,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:16:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:47,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:16:47,378 [219] [DEBUG] [app] Starting request: urn:request:d7e96204-370c-4c2f-a97c-3a209d2df93b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:47,378 [221] [DEBUG] [app] Starting request: urn:request:9010cde5-0a04-4d5d-a706-b33fa181dcfd (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:47,378 [219] [DEBUG] [app] Ending request: urn:request:d7e96204-370c-4c2f-a97c-3a209d2df93b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:47,378 [221] [DEBUG] [app] Ending request: urn:request:9010cde5-0a04-4d5d-a706-b33fa181dcfd (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:16:47,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:16:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:16:47,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:16:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:16:47,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:47,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:16:47,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:16:47,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:16:47,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:16:47,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:16:47,393 [221] [DEBUG] [app] Ending request: urn:request:8dbb4615-6dde-42bf-8d9a-e59974d585a1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:47,393 [222] [DEBUG] [app] Ending request: urn:request:dbf750a1-36e4-47d7-a622-bb82a3f52ead (/health/instance) gunicorn-web stdout | 2022-05-05 10:16:47,393 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:16:47,393 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:16:47,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:16:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:16:47,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:16:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:16:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:16:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-registry stdout | 2022-05-05 10:16:51,326 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:51,330 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:51,334 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:51,337 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:51,342 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:51,347 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:51,351 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:51,354 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:16:51,358 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:16:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:16:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:03.089985+00:00 (in 9.999502 seconds) notificationworker stdout | 2022-05-05 10:16:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:03 UTC)" (scheduled at 2022-05-05 10:16:53.089985+00:00) notificationworker stdout | 2022-05-05 10:16:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:16:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 16, 53, 91020), True, datetime.datetime(2022, 5, 5, 10, 16, 53, 91020), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:16:53,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:16:53,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:16:53,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:16:55,831 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:16:56,382 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:16:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:16:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:26.865082+00:00 (in 29.999516 seconds) gcworker stdout | 2022-05-05 10:16:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:26 UTC)" (scheduled at 2022-05-05 10:16:56.865082+00:00) gcworker stdout | 2022-05-05 10:16:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536216866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:16:56,876 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:16:56,876 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:16:56,876 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:16:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:16:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:28.201650+00:00 (in 29.999533 seconds) securityworker stdout | 2022-05-05 10:16:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:28 UTC)" (scheduled at 2022-05-05 10:16:58.201650+00:00) securityworker stdout | 2022-05-05 10:16:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:16:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:16:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:16:58,208 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:16:58,215 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:16:58,215 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:17:00,155 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:17:00,350 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:17:00,726 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:17:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:17:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:47.187465+00:00 (in 46.208062 seconds) exportactionlogsworker stdout | 2022-05-05 10:17:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:00 UTC)" (scheduled at 2022-05-05 10:17:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:17:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:17:00,980 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 0, 979881), True, datetime.datetime(2022, 5, 5, 10, 17, 0, 979881), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:17:00,990 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:17:00,990 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:17:00,990 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:00 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:17:01,614 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} chunkcleanupworker stdout | 2022-05-05 10:17:01,618 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} exportactionlogsworker stdout | 2022-05-05 10:17:01,957 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:17:02,369 [219] [DEBUG] [app] Starting request: urn:request:94742531-35d4-4d63-a5e2-4056a4cbe76c (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:02,369 [222] [DEBUG] [app] Starting request: urn:request:e162e2dc-061b-49e5-9c04-fee711fdf4e1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:02,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:02,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:17:02,374 [285] [DEBUG] [app] Starting request: urn:request:0a640a14-d521-46bf-9a8b-8e0ba014f8c5 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:02,374 [286] [DEBUG] [app] Starting request: urn:request:b917ed1f-36ca-4127-9975-a5bbec46f6fb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:02,376 [285] [DEBUG] [app] Ending request: urn:request:0a640a14-d521-46bf-9a8b-8e0ba014f8c5 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:02,376 [286] [DEBUG] [app] Ending request: urn:request:b917ed1f-36ca-4127-9975-a5bbec46f6fb (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.005 162 0.005) gunicorn-registry stdout | 2022-05-05 10:17:02,377 [285] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:17:02,377 [286] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:17:02,377 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:02,377 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:02,379 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:02,379 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:02,381 [219] [DEBUG] [app] Starting request: urn:request:a87d52ff-0483-406c-97d5-a4246967f810 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:02,381 [221] [DEBUG] [app] Starting request: urn:request:7888711e-ecf2-47ba-b0cb-d0b7b130e1bb (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:02,381 [219] [DEBUG] [app] Ending request: urn:request:a87d52ff-0483-406c-97d5-a4246967f810 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:02,381 [221] [DEBUG] [app] Ending request: urn:request:7888711e-ecf2-47ba-b0cb-d0b7b130e1bb (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:02,382 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:17:02,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:02,382 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:17:02,382 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:02,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:17:02,383 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:17:02,389 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:17:02,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:17:02,396 [222] [DEBUG] [app] Ending request: urn:request:e162e2dc-061b-49e5-9c04-fee711fdf4e1 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:02,396 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:17:02,396 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:17:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:17:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-web stdout | 2022-05-05 10:17:02,397 [219] [DEBUG] [app] Ending request: urn:request:94742531-35d4-4d63-a5e2-4056a4cbe76c (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:02,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:17:02,398 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:17:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:17:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.030) gunicorn-secscan stdout | 2022-05-05 10:17:03,089 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} notificationworker stdout | 2022-05-05 10:17:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:17:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:12.207990+00:00 (in 9.117481 seconds) notificationworker stdout | 2022-05-05 10:17:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:13 UTC)" (scheduled at 2022-05-05 10:17:03.089985+00:00) notificationworker stdout | 2022-05-05 10:17:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:17:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 3, 91068), True, datetime.datetime(2022, 5, 5, 10, 17, 3, 91068), 0, 'notification/%', 50, 1, 0]) gunicorn-secscan stdout | 2022-05-05 10:17:03,096 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:17:03,099 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} notificationworker stdout | 2022-05-05 10:17:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:17:03,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:17:03,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:13 UTC)" executed successfully manifestbackfillworker stdout | 2022-05-05 10:17:03,323 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:17:03,382 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:17:03,809 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:17:03,931 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:17:04,077 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:17:04,812 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:17:05,832 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:17:06,008 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:17:06,074 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:17:06,100 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:17:06,112 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:17:06,124 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:17:06,132 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:17:06,136 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:17:06,290 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:17:06,602 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:17:08,211 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:17:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:17:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:13.089985+00:00 (in 0.881417 seconds) notificationworker stdout | 2022-05-05 10:17:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:12 UTC)" (scheduled at 2022-05-05 10:17:12.207990+00:00) notificationworker stdout | 2022-05-05 10:17:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:17:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:17:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:17:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:23.089985+00:00 (in 9.999409 seconds) notificationworker stdout | 2022-05-05 10:17:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:23 UTC)" (scheduled at 2022-05-05 10:17:13.089985+00:00) notificationworker stdout | 2022-05-05 10:17:13,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:17:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 13, 91130), True, datetime.datetime(2022, 5, 5, 10, 17, 13, 91130), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:17:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:17:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:17:13,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:17:13,394 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:17:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:24.984990+00:00 (in 11.589886 seconds) repositorygcworker stdout | 2022-05-05 10:17:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:13 UTC)" (scheduled at 2022-05-05 10:17:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:17:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:17:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:17:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:17:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:38.977973+00:00 (in 24.690977 seconds) namespacegcworker stdout | 2022-05-05 10:17:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:14 UTC)" (scheduled at 2022-05-05 10:17:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:17:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:17:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:17:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:17:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:44.479974+00:00 (in 29.999504 seconds) buildlogsarchiver stdout | 2022-05-05 10:17:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:44 UTC)" (scheduled at 2022-05-05 10:17:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:17:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 17, 14, 480955), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:17:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:17:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:17:14,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:17:17,369 [222] [DEBUG] [app] Starting request: urn:request:cfe7f540-146a-490a-ae87-33b14e051c44 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:17,369 [221] [DEBUG] [app] Starting request: urn:request:7fc2806d-4310-4503-a16e-4c7c78c34f40 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:17,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:17:17,373 [288] [DEBUG] [app] Starting request: urn:request:af94e5a6-5414-42f7-a5fd-6e33d7f92d36 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:17,373 [292] [DEBUG] [app] Starting request: urn:request:5402ecf9-e4ad-4a8b-88b7-ade1348c5f4b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:17,374 [292] [DEBUG] [app] Ending request: urn:request:5402ecf9-e4ad-4a8b-88b7-ade1348c5f4b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:17,374 [288] [DEBUG] [app] Ending request: urn:request:af94e5a6-5414-42f7-a5fd-6e33d7f92d36 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:17,374 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:17:17,374 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:17:17,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:17,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:17,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:17,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:17,378 [222] [DEBUG] [app] Starting request: urn:request:2cd98fc0-5b53-4f6d-ba90-bd9091b6d2b2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:17,378 [219] [DEBUG] [app] Starting request: urn:request:c6f6742a-8eb1-40ce-b902-9046c6a5696b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:17,378 [222] [DEBUG] [app] Ending request: urn:request:2cd98fc0-5b53-4f6d-ba90-bd9091b6d2b2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:17,378 [219] [DEBUG] [app] Ending request: urn:request:c6f6742a-8eb1-40ce-b902-9046c6a5696b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:17,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:17:17,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:17:17,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:17,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:17,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:17:17,379 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:17:17,386 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:17:17,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:17:17,393 [222] [DEBUG] [app] Ending request: urn:request:cfe7f540-146a-490a-ae87-33b14e051c44 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:17,393 [221] [DEBUG] [app] Ending request: urn:request:7fc2806d-4310-4503-a16e-4c7c78c34f40 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:17,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:17:17,394 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:17:17,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:17:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:17:17,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:17:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:17:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.025) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:17:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-registry stdout | 2022-05-05 10:17:21,336 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:21,339 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:21,345 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:21,349 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:21,353 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:21,357 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:21,362 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:21,367 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:21,371 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:17:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:17:23,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:33.089985+00:00 (in 9.999136 seconds) notificationworker stdout | 2022-05-05 10:17:23,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:33 UTC)" (scheduled at 2022-05-05 10:17:23.089985+00:00) notificationworker stdout | 2022-05-05 10:17:23,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:17:23,093 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 23, 91947), True, datetime.datetime(2022, 5, 5, 10, 17, 23, 91947), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:17:23,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:17:23,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:17:23,104 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:17:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:17:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:00.385766+00:00 (in 35.400284 seconds) repositorygcworker stdout | 2022-05-05 10:17:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:24 UTC)" (scheduled at 2022-05-05 10:17:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:17:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:17:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 24, 985981), True, datetime.datetime(2022, 5, 5, 10, 17, 24, 985981), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:17:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:17:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:17:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:17:25,840 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:17:26,415 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:17:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:17:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:56.865082+00:00 (in 29.999532 seconds) gcworker stdout | 2022-05-05 10:17:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:56 UTC)" (scheduled at 2022-05-05 10:17:26.865082+00:00) gcworker stdout | 2022-05-05 10:17:26,866 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536246866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:17:26,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:17:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:17:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:17:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:17:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:58.201650+00:00 (in 29.999525 seconds) securityworker stdout | 2022-05-05 10:17:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:58 UTC)" (scheduled at 2022-05-05 10:17:28.201650+00:00) securityworker stdout | 2022-05-05 10:17:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:17:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:17:28,205 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:17:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:17:28,215 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:17:28,217 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:17:28,219 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:17:28,219 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:17:28,220 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 12, 28, 219739), 1, 2]) securityworker stdout | 2022-05-05 10:17:28,222 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:17:28,222 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:17:28,222 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 12, 28, 222235), 1, 2]) securityworker stdout | 2022-05-05 10:17:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:17:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:17:28,225 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:17:28,225 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:17:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:17:30,194 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:17:30,358 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:17:30,758 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} chunkcleanupworker stdout | 2022-05-05 10:17:31,636 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:17:31,649 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} securityscanningnotificationworker stdout | 2022-05-05 10:17:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:17:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:35.969483+00:00 (in 4.194983 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:17:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:31 UTC)" (scheduled at 2022-05-05 10:17:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:17:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:17:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 31, 775065), True, datetime.datetime(2022, 5, 5, 10, 17, 31, 775065), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:17:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:17:31,787 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:17:31,787 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:31 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:17:31,965 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:17:32,370 [221] [DEBUG] [app] Starting request: urn:request:fb98db04-c257-40fe-befe-f41baf800e67 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:32,370 [222] [DEBUG] [app] Starting request: urn:request:bfe8554c-1572-455a-8815-4cef2e7da542 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:32,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:32,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:17:32,373 [288] [DEBUG] [app] Starting request: urn:request:56794aad-1f81-4023-b7fa-23b840532544 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:32,374 [288] [DEBUG] [app] Ending request: urn:request:56794aad-1f81-4023-b7fa-23b840532544 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:32,375 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:17:32,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:17:32,375 [291] [DEBUG] [app] Starting request: urn:request:4ab63352-2b3d-4e7d-b61f-def512cfa07e (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:32,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:17:32,377 [291] [DEBUG] [app] Ending request: urn:request:4ab63352-2b3d-4e7d-b61f-def512cfa07e (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.006) gunicorn-registry stdout | 2022-05-05 10:17:32,378 [291] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:17:32,378 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:32,379 [221] [DEBUG] [app] Starting request: urn:request:0e995c47-4099-45ca-8fb2-ea1b9f9e2a47 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:32,379 [221] [DEBUG] [app] Ending request: urn:request:0e995c47-4099-45ca-8fb2-ea1b9f9e2a47 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:17:32,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:17:32,380 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:32,381 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:32,381 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:17:32,382 [219] [DEBUG] [app] Starting request: urn:request:2a51f36e-b5dd-465b-ba36-10ec705fc041 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:32,383 [219] [DEBUG] [app] Ending request: urn:request:2a51f36e-b5dd-465b-ba36-10ec705fc041 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:32,383 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:17:32,383 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:32,384 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:17:32,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:17:32,389 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:17:32,394 [221] [DEBUG] [app] Ending request: urn:request:fb98db04-c257-40fe-befe-f41baf800e67 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:32,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:17:32,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:17:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:17:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:17:32,397 [222] [DEBUG] [app] Ending request: urn:request:bfe8554c-1572-455a-8815-4cef2e7da542 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:32,397 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:17:32,397 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:17:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:17:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.029) notificationworker stdout | 2022-05-05 10:17:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:17:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:43.089985+00:00 (in 9.999229 seconds) notificationworker stdout | 2022-05-05 10:17:33,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:43 UTC)" (scheduled at 2022-05-05 10:17:33.089985+00:00) notificationworker stdout | 2022-05-05 10:17:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:17:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 33, 91244), True, datetime.datetime(2022, 5, 5, 10, 17, 33, 91244), 0, 'notification/%', 50, 1, 0]) gunicorn-secscan stdout | 2022-05-05 10:17:33,098 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} notificationworker stdout | 2022-05-05 10:17:33,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:17:33,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:17:33,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:43 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:17:33,107 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:17:33,110 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:17:33,350 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:17:33,419 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:17:33,822 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:17:33,965 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:17:34,108 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:17:34,822 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:17:35,866 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} securityscanningnotificationworker stdout | 2022-05-05 10:17:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:17:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:03.965762+00:00 (in 27.995684 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:17:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:35 UTC)" (scheduled at 2022-05-05 10:17:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:17:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:17:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:35 UTC)" executed successfully servicekey stdout | 2022-05-05 10:17:36,022 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:17:36,082 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:17:36,108 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:17:36,121 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:17:36,132 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:17:36,142 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:17:36,145 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:17:36,321 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:17:36,620 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:17:38,218 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:17:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:17:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:14.286471+00:00 (in 35.307962 seconds) namespacegcworker stdout | 2022-05-05 10:17:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:38 UTC)" (scheduled at 2022-05-05 10:17:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:17:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:17:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 38, 979160), True, datetime.datetime(2022, 5, 5, 10, 17, 38, 979160), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:17:38,990 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:17:38,990 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:17:38,990 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:17:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:17:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:17:53.089985+00:00 (in 9.999492 seconds) notificationworker stdout | 2022-05-05 10:17:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:53 UTC)" (scheduled at 2022-05-05 10:17:43.089985+00:00) notificationworker stdout | 2022-05-05 10:17:43,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:17:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 43, 91001), True, datetime.datetime(2022, 5, 5, 10, 17, 43, 91001), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:17:43,107 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:17:43,107 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:17:43,107 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:17:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:17:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:17:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:14.479974+00:00 (in 29.999506 seconds) buildlogsarchiver stdout | 2022-05-05 10:17:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:14 UTC)" (scheduled at 2022-05-05 10:17:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:17:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 17, 44, 480941), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:17:44,494 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:17:44,494 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:17:44,494 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:17:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:17:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:00.978909+00:00 (in 13.790981 seconds) exportactionlogsworker stdout | 2022-05-05 10:17:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:47 UTC)" (scheduled at 2022-05-05 10:17:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:17:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:17:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:18:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:17:47,370 [219] [DEBUG] [app] Starting request: urn:request:4a45dfa9-44d5-4c13-ab14-5d6d4aa8cadf (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:47,370 [222] [DEBUG] [app] Starting request: urn:request:ba1e2ee0-8223-4c7c-a8a6-5266ba62afb7 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:47,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:17:47,373 [288] [DEBUG] [app] Starting request: urn:request:7161e050-8624-4412-a89c-305d15e4c7e5 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:47,374 [292] [DEBUG] [app] Starting request: urn:request:159e80ba-4e05-44b7-8a20-24b88d3a66a0 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:47,374 [288] [DEBUG] [app] Ending request: urn:request:7161e050-8624-4412-a89c-305d15e4c7e5 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:47,374 [292] [DEBUG] [app] Ending request: urn:request:159e80ba-4e05-44b7-8a20-24b88d3a66a0 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:17:47,374 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:17:47,375 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:17:47,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:47,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:47,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:47,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:17:47,378 [219] [DEBUG] [app] Starting request: urn:request:901ccec7-4da5-4b39-a546-75889195610b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:47,379 [220] [DEBUG] [app] Starting request: urn:request:4d216d20-9b81-4518-be53-9dbe6cf5706f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:47,379 [219] [DEBUG] [app] Ending request: urn:request:901ccec7-4da5-4b39-a546-75889195610b (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:17:47,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:17:47,379 [220] [DEBUG] [app] Ending request: urn:request:4d216d20-9b81-4518-be53-9dbe6cf5706f (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:17:47,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:47,380 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:17:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:17:47,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql nginx stdout | 127.0.0.1 () - - [05/May/2022:10:17:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:17:47,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:17:47,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:17:47,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:17:47,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:17:47,396 [219] [DEBUG] [app] Ending request: urn:request:4a45dfa9-44d5-4c13-ab14-5d6d4aa8cadf (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:47,397 [222] [DEBUG] [app] Ending request: urn:request:ba1e2ee0-8223-4c7c-a8a6-5266ba62afb7 (/health/instance) gunicorn-web stdout | 2022-05-05 10:17:47,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:17:47,397 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:17:47,397 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:17:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:17:47,397 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:17:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:17:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:17:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-registry stdout | 2022-05-05 10:17:51,346 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:51,351 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:51,357 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:51,362 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:51,368 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:51,372 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:51,377 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:51,381 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:17:51,387 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:17:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:17:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:03.089985+00:00 (in 9.999442 seconds) notificationworker stdout | 2022-05-05 10:17:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:03 UTC)" (scheduled at 2022-05-05 10:17:53.089985+00:00) notificationworker stdout | 2022-05-05 10:17:53,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:17:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 17, 53, 91027), True, datetime.datetime(2022, 5, 5, 10, 17, 53, 91027), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:17:53,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:17:53,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:17:53,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:17:55,876 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:17:56,450 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:17:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:17:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:26.865082+00:00 (in 29.999493 seconds) gcworker stdout | 2022-05-05 10:17:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:26 UTC)" (scheduled at 2022-05-05 10:17:56.865082+00:00) gcworker stdout | 2022-05-05 10:17:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536276866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:17:56,879 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:17:56,879 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:17:56,879 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:17:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:17:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:28.201650+00:00 (in 29.999459 seconds) securityworker stdout | 2022-05-05 10:17:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:28 UTC)" (scheduled at 2022-05-05 10:17:58.201650+00:00) securityworker stdout | 2022-05-05 10:17:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:17:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:17:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:17:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:17:58,215 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:17:58,216 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:18:00,226 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:18:00,375 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} repositorygcworker stdout | 2022-05-05 10:18:00,386 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:18:00,386 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:13.394619+00:00 (in 13.008374 seconds) repositorygcworker stdout | 2022-05-05 10:18:00,386 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:23:00 UTC)" (scheduled at 2022-05-05 10:18:00.385766+00:00) repositorygcworker stdout | 2022-05-05 10:18:00,387 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 18, 0, 386681), 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:18:00,396 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 18, 0, 386681), True, datetime.datetime(2022, 5, 5, 10, 18, 0, 386681), 0, 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:18:00,399 [93] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 18, 0, 386681), True, datetime.datetime(2022, 5, 5, 10, 18, 0, 386681), 0, 'repositorygc/%', False, datetime.datetime(2022, 5, 5, 10, 18, 0, 386681), 'repositorygc/%']) repositorygcworker stdout | 2022-05-05 10:18:00,402 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:18:00,402 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:23:00 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:18:00,776 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:18:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:18:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:47.187465+00:00 (in 46.208084 seconds) exportactionlogsworker stdout | 2022-05-05 10:18:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:00 UTC)" (scheduled at 2022-05-05 10:18:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:18:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:18:00,981 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 0, 979851), True, datetime.datetime(2022, 5, 5, 10, 18, 0, 979851), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:18:00,991 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:18:00,991 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:18:00,991 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:18:01,653 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:18:01,686 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:18:02,002 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:18:02,370 [219] [DEBUG] [app] Starting request: urn:request:eddc79d0-157a-4a64-b9e9-adea45dffea0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:02,370 [221] [DEBUG] [app] Starting request: urn:request:57791958-fb39-42da-af95-7be262c034a5 (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:02,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:02,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:18:02,374 [288] [DEBUG] [app] Starting request: urn:request:c3e102cb-8a77-48ea-b3b6-8357a425e636 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:02,374 [292] [DEBUG] [app] Starting request: urn:request:9a2fd120-82c9-4665-a859-317d6a315ced (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:02,375 [292] [DEBUG] [app] Ending request: urn:request:9a2fd120-82c9-4665-a859-317d6a315ced (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:02,375 [288] [DEBUG] [app] Ending request: urn:request:c3e102cb-8a77-48ea-b3b6-8357a425e636 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:02,375 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:18:02,375 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-web stdout | 2022-05-05 10:18:02,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:02,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:02,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:02,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:02,379 [219] [DEBUG] [app] Starting request: urn:request:a6f82145-5a78-45eb-93d8-938c0151687b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:02,379 [220] [DEBUG] [app] Starting request: urn:request:cbf18e02-bc28-4f86-b988-c55ac690318a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:02,380 [219] [DEBUG] [app] Ending request: urn:request:a6f82145-5a78-45eb-93d8-938c0151687b (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:02,380 [220] [DEBUG] [app] Ending request: urn:request:cbf18e02-bc28-4f86-b988-c55ac690318a (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:18:02,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:02,380 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:02,380 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:02,380 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:02,381 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:18:02,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:18:02,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:18:02,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:18:02,394 [221] [DEBUG] [app] Ending request: urn:request:57791958-fb39-42da-af95-7be262c034a5 (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:02,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:18:02,394 [219] [DEBUG] [app] Ending request: urn:request:eddc79d0-157a-4a64-b9e9-adea45dffea0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:02,394 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:18:02,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:18:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:18:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:18:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:18:02,394 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:18:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" notificationworker stdout | 2022-05-05 10:18:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:18:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:12.207990+00:00 (in 9.117501 seconds) notificationworker stdout | 2022-05-05 10:18:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:13 UTC)" (scheduled at 2022-05-05 10:18:03.089985+00:00) notificationworker stdout | 2022-05-05 10:18:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:18:03,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 3, 90978), True, datetime.datetime(2022, 5, 5, 10, 18, 3, 90978), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:18:03,100 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:18:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:18:03,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:13 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:18:03,106 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:18:03,116 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:18:03,120 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:18:03,379 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:18:03,455 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:18:03,859 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} securityscanningnotificationworker stdout | 2022-05-05 10:18:03,966 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:18:03,966 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:31.774027+00:00 (in 27.807639 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:18:03,967 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:23:03 UTC)" (scheduled at 2022-05-05 10:18:03.965762+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:18:03,968 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 18, 3, 967505), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:18:03,977 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 18, 3, 967505), True, datetime.datetime(2022, 5, 5, 10, 18, 3, 967505), 0, 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:18:03,980 [94] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 18, 3, 967505), True, datetime.datetime(2022, 5, 5, 10, 18, 3, 967505), 0, 'secscanv4/%', False, datetime.datetime(2022, 5, 5, 10, 18, 3, 967505), 'secscanv4/%']) securityscanningnotificationworker stdout | 2022-05-05 10:18:03,982 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:18:03,982 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:23:03 UTC)" executed successfully logrotateworker stdout | 2022-05-05 10:18:03,990 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:18:04,135 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:18:04,852 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:18:05,899 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:18:06,035 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:18:06,117 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:18:06,120 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:18:06,130 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:18:06,140 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:18:06,150 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:18:06,154 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:18:06,358 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:18:06,657 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:18:08,233 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:18:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:18:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:13.089985+00:00 (in 0.881490 seconds) notificationworker stdout | 2022-05-05 10:18:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:12 UTC)" (scheduled at 2022-05-05 10:18:12.207990+00:00) notificationworker stdout | 2022-05-05 10:18:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:18:12,208 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:18:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:18:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:23.089985+00:00 (in 9.999515 seconds) notificationworker stdout | 2022-05-05 10:18:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:23 UTC)" (scheduled at 2022-05-05 10:18:13.089985+00:00) notificationworker stdout | 2022-05-05 10:18:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:18:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 13, 90951), True, datetime.datetime(2022, 5, 5, 10, 18, 13, 90951), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:18:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:18:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:18:13,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:18:13,394 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:18:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:24.984990+00:00 (in 11.589908 seconds) repositorygcworker stdout | 2022-05-05 10:18:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:13 UTC)" (scheduled at 2022-05-05 10:18:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:18:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:18:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:18:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:18:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:38.977973+00:00 (in 24.690941 seconds) namespacegcworker stdout | 2022-05-05 10:18:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:14 UTC)" (scheduled at 2022-05-05 10:18:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:18:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:18:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:18:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:18:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:44.479974+00:00 (in 29.999468 seconds) buildlogsarchiver stdout | 2022-05-05 10:18:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:44 UTC)" (scheduled at 2022-05-05 10:18:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:18:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 18, 14, 481067), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:18:14,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:18:14,492 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:18:14,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:18:17,370 [219] [DEBUG] [app] Starting request: urn:request:068b8d28-b8cd-43b4-9737-7ca88c271d3a (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:17,370 [222] [DEBUG] [app] Starting request: urn:request:d644970c-5622-41e6-ab46-757cc3ba5d9f (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:17,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:18:17,373 [292] [DEBUG] [app] Starting request: urn:request:f3f75d2c-7476-4d92-9008-7ea86c947afb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:17,374 [292] [DEBUG] [app] Ending request: urn:request:f3f75d2c-7476-4d92-9008-7ea86c947afb (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:18:17,374 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:17,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-registry stdout | 2022-05-05 10:18:17,375 [289] [DEBUG] [app] Starting request: urn:request:3c9cc346-1620-49ba-be48-489a495f47ec (/v1/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:17,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:18:17,377 [289] [DEBUG] [app] Ending request: urn:request:3c9cc346-1620-49ba-be48-489a495f47ec (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.006 162 0.005) gunicorn-registry stdout | 2022-05-05 10:18:17,378 [289] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:17,378 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:17,379 [221] [DEBUG] [app] Starting request: urn:request:4e03287c-f180-4a31-bf5e-98de3fda403c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:17,379 [221] [DEBUG] [app] Ending request: urn:request:4e03287c-f180-4a31-bf5e-98de3fda403c (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:17,380 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:17,380 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:17,380 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:18:17,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:18:17,382 [221] [DEBUG] [app] Starting request: urn:request:dd4a995e-2caf-44da-bfbb-9220a90727c2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:17,382 [221] [DEBUG] [app] Ending request: urn:request:dd4a995e-2caf-44da-bfbb-9220a90727c2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:17,383 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:17,383 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:18:17,383 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:18:17,388 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:18:17,389 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:18:17,398 [219] [DEBUG] [app] Ending request: urn:request:068b8d28-b8cd-43b4-9737-7ca88c271d3a (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:17,398 [222] [DEBUG] [app] Ending request: urn:request:d644970c-5622-41e6-ab46-757cc3ba5d9f (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:17,398 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:18:17,398 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:18:17,398 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:18:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:18:17,398 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:18:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:18:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:18:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) gunicorn-registry stdout | 2022-05-05 10:18:21,360 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:21,364 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:21,370 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:21,374 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:21,379 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:21,384 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:21,388 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:21,392 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:21,396 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:18:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:18:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:33.089985+00:00 (in 9.999514 seconds) notificationworker stdout | 2022-05-05 10:18:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:33 UTC)" (scheduled at 2022-05-05 10:18:23.089985+00:00) notificationworker stdout | 2022-05-05 10:18:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:18:23,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 23, 90940), True, datetime.datetime(2022, 5, 5, 10, 18, 23, 90940), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:18:23,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:18:23,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:18:23,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:18:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:18:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:13.394619+00:00 (in 48.409155 seconds) repositorygcworker stdout | 2022-05-05 10:18:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:24 UTC)" (scheduled at 2022-05-05 10:18:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:18:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:18:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 24, 986013), True, datetime.datetime(2022, 5, 5, 10, 18, 24, 986013), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:18:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:18:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:18:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:18:25,883 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:18:26,480 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:18:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:18:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:56.865082+00:00 (in 29.999491 seconds) gcworker stdout | 2022-05-05 10:18:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:56 UTC)" (scheduled at 2022-05-05 10:18:26.865082+00:00) gcworker stdout | 2022-05-05 10:18:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536306866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:18:26,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:18:26,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:18:26,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:18:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:18:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:58.201650+00:00 (in 29.999469 seconds) securityworker stdout | 2022-05-05 10:18:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:58 UTC)" (scheduled at 2022-05-05 10:18:28.201650+00:00) securityworker stdout | 2022-05-05 10:18:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:18:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:18:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:18:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:18:28,215 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:18:28,217 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:18:28,220 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:18:28,220 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:18:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 13, 28, 220534), 1, 2]) securityworker stdout | 2022-05-05 10:18:28,222 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:18:28,222 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:18:28,223 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 13, 28, 223030), 1, 2]) securityworker stdout | 2022-05-05 10:18:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:18:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:18:28,225 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:18:28,225 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:18:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:18:30,256 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:18:30,392 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:18:30,813 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} chunkcleanupworker stdout | 2022-05-05 10:18:31,685 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:18:31,706 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} securityscanningnotificationworker stdout | 2022-05-05 10:18:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:18:31,775 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:35.969483+00:00 (in 4.194519 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:18:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:31 UTC)" (scheduled at 2022-05-05 10:18:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:18:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:18:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 31, 775533), True, datetime.datetime(2022, 5, 5, 10, 18, 31, 775533), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:18:31,788 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:18:31,789 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:18:31,789 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:31 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:18:32,011 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:18:32,369 [222] [DEBUG] [app] Starting request: urn:request:a6449fb9-cafa-4950-b5bc-6db08b57c024 (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:32,369 [221] [DEBUG] [app] Starting request: urn:request:8f0bd186-3af0-4d7c-94df-29d5e76f07fc (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:32,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:32,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:18:32,373 [288] [DEBUG] [app] Starting request: urn:request:b0d680e5-e84a-4295-84bd-2b435c334800 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:32,373 [288] [DEBUG] [app] Ending request: urn:request:b0d680e5-e84a-4295-84bd-2b435c334800 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:32,373 [291] [DEBUG] [app] Starting request: urn:request:09463649-8c7a-496f-9672-341e4716fb0f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:32,374 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:32,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:18:32,374 [291] [DEBUG] [app] Ending request: urn:request:09463649-8c7a-496f-9672-341e4716fb0f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:32,375 [291] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:32,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:18:32,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:32,377 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:32,378 [219] [DEBUG] [app] Starting request: urn:request:40e06b80-f999-413b-8a78-c046fd9edfa5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:32,378 [219] [DEBUG] [app] Ending request: urn:request:40e06b80-f999-413b-8a78-c046fd9edfa5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:32,378 [221] [DEBUG] [app] Starting request: urn:request:be63aaf2-07f8-4499-b740-e95f05aa6ac7 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:32,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:32,379 [221] [DEBUG] [app] Ending request: urn:request:be63aaf2-07f8-4499-b740-e95f05aa6ac7 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:18:32,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:32,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:32,380 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:32,380 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:18:32,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:18:32,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:18:32,389 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:18:32,393 [222] [DEBUG] [app] Ending request: urn:request:a6449fb9-cafa-4950-b5bc-6db08b57c024 (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:32,393 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:18:32,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:18:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:18:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.025) gunicorn-web stdout | 2022-05-05 10:18:32,398 [221] [DEBUG] [app] Ending request: urn:request:8f0bd186-3af0-4d7c-94df-29d5e76f07fc (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:32,398 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:18:32,398 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:18:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:18:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.030) notificationworker stdout | 2022-05-05 10:18:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:18:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:43.089985+00:00 (in 9.999491 seconds) notificationworker stdout | 2022-05-05 10:18:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:43 UTC)" (scheduled at 2022-05-05 10:18:33.089985+00:00) notificationworker stdout | 2022-05-05 10:18:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:18:33,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 33, 91043), True, datetime.datetime(2022, 5, 5, 10, 18, 33, 91043), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:18:33,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:18:33,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:18:33,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:43 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:18:33,115 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:18:33,124 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:18:33,129 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:18:33,416 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:18:33,490 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:18:33,893 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:18:34,005 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:18:34,170 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:18:34,860 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:18:35,909 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} securityscanningnotificationworker stdout | 2022-05-05 10:18:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:18:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:31.774027+00:00 (in 55.803829 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:18:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:35 UTC)" (scheduled at 2022-05-05 10:18:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:18:35,971 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:18:35,971 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:35 UTC)" executed successfully servicekey stdout | 2022-05-05 10:18:36,071 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} gunicorn-web stdout | 2022-05-05 10:18:36,128 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:18:36,138 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:18:36,147 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} globalpromstats stdout | 2022-05-05 10:18:36,152 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} gunicorn-web stdout | 2022-05-05 10:18:36,158 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:18:36,162 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} notificationworker stdout | 2022-05-05 10:18:36,386 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:18:36,675 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:18:38,253 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:18:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:18:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:14.286471+00:00 (in 35.308035 seconds) namespacegcworker stdout | 2022-05-05 10:18:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:38 UTC)" (scheduled at 2022-05-05 10:18:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:18:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:18:38,979 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 38, 978919), True, datetime.datetime(2022, 5, 5, 10, 18, 38, 978919), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:18:38,989 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:18:38,989 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:18:38,989 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:18:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:18:43,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:18:53.089985+00:00 (in 9.999221 seconds) notificationworker stdout | 2022-05-05 10:18:43,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:53 UTC)" (scheduled at 2022-05-05 10:18:43.089985+00:00) notificationworker stdout | 2022-05-05 10:18:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:18:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 43, 91533), True, datetime.datetime(2022, 5, 5, 10, 18, 43, 91533), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:18:43,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:18:43,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:18:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:18:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:18:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:18:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:14.479974+00:00 (in 29.999422 seconds) buildlogsarchiver stdout | 2022-05-05 10:18:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:14 UTC)" (scheduled at 2022-05-05 10:18:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:18:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 18, 44, 481035), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:18:44,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:18:44,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:18:44,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:18:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:18:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:00.978909+00:00 (in 13.790939 seconds) exportactionlogsworker stdout | 2022-05-05 10:18:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:47 UTC)" (scheduled at 2022-05-05 10:18:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:18:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:18:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:19:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:18:47,370 [219] [DEBUG] [app] Starting request: urn:request:31fa2632-8b08-40fa-9a75-9d0f663fc7ea (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:47,370 [222] [DEBUG] [app] Starting request: urn:request:e3851759-cee2-4524-98e4-c82f6c6fb4c5 (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:47,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:47,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:18:47,374 [292] [DEBUG] [app] Starting request: urn:request:f7852d5b-38b2-47fd-9c53-c8b378fc779a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:47,374 [291] [DEBUG] [app] Starting request: urn:request:05c3cca6-13d6-4965-8573-c234c5fffc93 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:47,375 [291] [DEBUG] [app] Ending request: urn:request:05c3cca6-13d6-4965-8573-c234c5fffc93 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:47,375 [292] [DEBUG] [app] Ending request: urn:request:f7852d5b-38b2-47fd-9c53-c8b378fc779a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:18:47,376 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:18:47,376 [291] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:47,376 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:47,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-web stdout | 2022-05-05 10:18:47,378 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:47,378 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:18:47,379 [219] [DEBUG] [app] Starting request: urn:request:bdd64b42-4b63-48c7-8bdb-105ea6a510e6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:47,380 [220] [DEBUG] [app] Starting request: urn:request:7bd90bc8-4030-4500-bdf8-198c540e937d (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:47,380 [219] [DEBUG] [app] Ending request: urn:request:bdd64b42-4b63-48c7-8bdb-105ea6a510e6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:18:47,380 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:47,380 [220] [DEBUG] [app] Ending request: urn:request:7bd90bc8-4030-4500-bdf8-198c540e937d (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:18:47,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:18:47,381 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:18:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:18:47,381 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:18:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:18:47,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:18:47,381 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:18:47,388 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:18:47,389 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:18:47,395 [219] [DEBUG] [app] Ending request: urn:request:31fa2632-8b08-40fa-9a75-9d0f663fc7ea (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:47,395 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:18:47,395 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:18:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:18:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:18:47,398 [222] [DEBUG] [app] Ending request: urn:request:e3851759-cee2-4524-98e4-c82f6c6fb4c5 (/health/instance) gunicorn-web stdout | 2022-05-05 10:18:47,398 [222] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:18:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.030) gunicorn-web stdout | 2022-05-05 10:18:47,399 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:18:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:18:51,374 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:51,379 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:51,383 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:51,386 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:51,390 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:51,396 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:51,401 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:51,405 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:18:51,408 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:18:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:18:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:03.089985+00:00 (in 9.999211 seconds) notificationworker stdout | 2022-05-05 10:18:53,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:03 UTC)" (scheduled at 2022-05-05 10:18:53.089985+00:00) notificationworker stdout | 2022-05-05 10:18:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:18:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 18, 53, 91354), True, datetime.datetime(2022, 5, 5, 10, 18, 53, 91354), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:18:53,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:18:53,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:18:53,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:18:55,919 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:18:56,513 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:18:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:18:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:26.865082+00:00 (in 29.999518 seconds) gcworker stdout | 2022-05-05 10:18:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:26 UTC)" (scheduled at 2022-05-05 10:18:56.865082+00:00) gcworker stdout | 2022-05-05 10:18:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536336866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:18:56,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:18:56,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:18:56,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:18:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:18:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:28.201650+00:00 (in 29.999530 seconds) securityworker stdout | 2022-05-05 10:18:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:28 UTC)" (scheduled at 2022-05-05 10:18:58.201650+00:00) securityworker stdout | 2022-05-05 10:18:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:18:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:18:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:18:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:18:58,217 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:18:58,217 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:19:00,292 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:19:00,422 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:19:00,843 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:19:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:19:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:47.187465+00:00 (in 46.207962 seconds) exportactionlogsworker stdout | 2022-05-05 10:19:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:00 UTC)" (scheduled at 2022-05-05 10:19:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:19:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:19:00,980 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 0, 979985), True, datetime.datetime(2022, 5, 5, 10, 19, 0, 979985), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:19:00,997 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:19:00,997 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:19:00,997 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:19:01,722 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:19:01,743 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:19:02,026 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:19:02,369 [222] [DEBUG] [app] Starting request: urn:request:ca03ae46-9fc0-47b9-9047-7831a192720b (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:02,369 [220] [DEBUG] [app] Starting request: urn:request:40e27fd8-5246-4f0c-aba8-a5195d8ad5e0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:02,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:02,371 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:19:02,372 [291] [DEBUG] [app] Starting request: urn:request:b7f62f5f-8c1c-4653-b20e-8d846b613beb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:02,373 [288] [DEBUG] [app] Starting request: urn:request:e8d5a9a3-1916-465a-9a46-6cba4958e56f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:02,373 [291] [DEBUG] [app] Ending request: urn:request:b7f62f5f-8c1c-4653-b20e-8d846b613beb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:02,373 [288] [DEBUG] [app] Ending request: urn:request:e8d5a9a3-1916-465a-9a46-6cba4958e56f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:02,374 [291] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:19:02,374 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:19:02,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:02,374 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:02,375 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:02,375 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:02,377 [220] [DEBUG] [app] Starting request: urn:request:799c55f1-99aa-470e-861f-d4a1ef7b71e9 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:02,377 [219] [DEBUG] [app] Starting request: urn:request:7aa7f2b5-29e3-4580-b01f-51b32302a988 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:02,377 [220] [DEBUG] [app] Ending request: urn:request:799c55f1-99aa-470e-861f-d4a1ef7b71e9 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:02,378 [219] [DEBUG] [app] Ending request: urn:request:7aa7f2b5-29e3-4580-b01f-51b32302a988 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:02,378 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:19:02,378 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:02,378 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:19:02,378 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:19:02,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:19:02,379 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:19:02,385 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:19:02,386 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:19:02,392 [220] [DEBUG] [app] Ending request: urn:request:40e27fd8-5246-4f0c-aba8-a5195d8ad5e0 (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:02,392 [222] [DEBUG] [app] Ending request: urn:request:ca03ae46-9fc0-47b9-9047-7831a192720b (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:02,392 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:02,392 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:02,393 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:19:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:19:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.024 118 0.025) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:19:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:19:02,393 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:19:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" notificationworker stdout | 2022-05-05 10:19:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:19:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:12.207990+00:00 (in 9.117526 seconds) notificationworker stdout | 2022-05-05 10:19:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:13 UTC)" (scheduled at 2022-05-05 10:19:03.089985+00:00) notificationworker stdout | 2022-05-05 10:19:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:19:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 3, 91033), True, datetime.datetime(2022, 5, 5, 10, 19, 3, 91033), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:19:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:19:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:19:03,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:13 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:19:03,124 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:19:03,134 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:19:03,137 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:19:03,450 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:19:03,509 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:19:03,929 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:19:04,042 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:19:04,181 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:19:04,900 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:19:05,946 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:19:06,106 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} gunicorn-web stdout | 2022-05-05 10:19:06,136 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:19:06,146 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:19:06,157 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:19:06,166 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:19:06,170 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} globalpromstats stdout | 2022-05-05 10:19:06,181 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:19:06,411 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:19:06,711 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:19:08,289 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:19:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:19:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:13.089985+00:00 (in 0.881394 seconds) notificationworker stdout | 2022-05-05 10:19:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:12 UTC)" (scheduled at 2022-05-05 10:19:12.207990+00:00) notificationworker stdout | 2022-05-05 10:19:12,209 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:19:12,209 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:19:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:19:13,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:23.089985+00:00 (in 9.999233 seconds) notificationworker stdout | 2022-05-05 10:19:13,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:23 UTC)" (scheduled at 2022-05-05 10:19:13.089985+00:00) notificationworker stdout | 2022-05-05 10:19:13,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:19:13,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 13, 91569), True, datetime.datetime(2022, 5, 5, 10, 19, 13, 91569), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:19:13,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:19:13,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:19:13,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:19:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:19:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:24.984990+00:00 (in 11.589813 seconds) repositorygcworker stdout | 2022-05-05 10:19:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:13 UTC)" (scheduled at 2022-05-05 10:19:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:19:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:19:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:19:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:19:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:23.280939+00:00 (in 8.993997 seconds) namespacegcworker stdout | 2022-05-05 10:19:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:14 UTC)" (scheduled at 2022-05-05 10:19:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:19:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:19:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:19:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:19:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:44.479974+00:00 (in 29.999462 seconds) buildlogsarchiver stdout | 2022-05-05 10:19:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:44 UTC)" (scheduled at 2022-05-05 10:19:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:19:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 19, 14, 481017), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:19:14,491 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:19:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:19:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:19:17,369 [219] [DEBUG] [app] Starting request: urn:request:645aa6f0-412f-4f8a-8bd0-3281110e8e96 (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:17,369 [221] [DEBUG] [app] Starting request: urn:request:9e5004e3-ecd9-4e37-9bdc-817ba7b61fda (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:17,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:17,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:19:17,372 [288] [DEBUG] [app] Starting request: urn:request:51e5ab47-df8d-43dd-9d4e-f8e2db982968 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:17,372 [290] [DEBUG] [app] Starting request: urn:request:f6bed4e3-1022-4346-834e-ec583806de6b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:17,373 [290] [DEBUG] [app] Ending request: urn:request:f6bed4e3-1022-4346-834e-ec583806de6b (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:17,373 [288] [DEBUG] [app] Ending request: urn:request:51e5ab47-df8d-43dd-9d4e-f8e2db982968 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:19:17,373 [290] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:19:17,373 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:19:17,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:17,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:17,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:17,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:17,377 [221] [DEBUG] [app] Starting request: urn:request:5677d653-cc57-4f29-ad7f-bfe45db45c8e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:17,377 [222] [DEBUG] [app] Starting request: urn:request:5de24cfa-8589-4082-af2a-c1ada66c7fbd (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:17,378 [221] [DEBUG] [app] Ending request: urn:request:5677d653-cc57-4f29-ad7f-bfe45db45c8e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:17,378 [222] [DEBUG] [app] Ending request: urn:request:5de24cfa-8589-4082-af2a-c1ada66c7fbd (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:17,378 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:19:17,378 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:19:17,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:17,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 159 0.002) gunicorn-web stdout | 2022-05-05 10:19:17,379 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:19:17,379 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:19:17,385 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:19:17,386 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:19:17,392 [221] [DEBUG] [app] Ending request: urn:request:9e5004e3-ecd9-4e37-9bdc-817ba7b61fda (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:17,392 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:17,393 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:19:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:19:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:19:17,393 [219] [DEBUG] [app] Ending request: urn:request:645aa6f0-412f-4f8a-8bd0-3281110e8e96 (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:17,394 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:17,394 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:19:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:19:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-registry stdout | 2022-05-05 10:19:21,384 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:21,391 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:21,395 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:21,399 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:21,402 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:21,407 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:21,413 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:21,417 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:21,420 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:19:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:19:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:33.089985+00:00 (in 9.999186 seconds) notificationworker stdout | 2022-05-05 10:19:23,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:33 UTC)" (scheduled at 2022-05-05 10:19:23.089985+00:00) notificationworker stdout | 2022-05-05 10:19:23,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:19:23,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 23, 91416), True, datetime.datetime(2022, 5, 5, 10, 19, 23, 91416), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:19:23,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:19:23,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:19:23,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:33 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:19:23,281 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:19:23,281 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:38.977973+00:00 (in 15.696535 seconds) namespacegcworker stdout | 2022-05-05 10:19:23,281 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:24:23 UTC)" (scheduled at 2022-05-05 10:19:23.280939+00:00) namespacegcworker stdout | 2022-05-05 10:19:23,282 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 19, 23, 281821), 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:19:23,293 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 19, 23, 281821), True, datetime.datetime(2022, 5, 5, 10, 19, 23, 281821), 0, 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:19:23,296 [71] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 19, 23, 281821), True, datetime.datetime(2022, 5, 5, 10, 19, 23, 281821), 0, 'namespacegc/%', False, datetime.datetime(2022, 5, 5, 10, 19, 23, 281821), 'namespacegc/%']) namespacegcworker stdout | 2022-05-05 10:19:23,298 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:19:23,298 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:24:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:19:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:19:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:13.394619+00:00 (in 48.409169 seconds) repositorygcworker stdout | 2022-05-05 10:19:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:24 UTC)" (scheduled at 2022-05-05 10:19:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:19:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:19:24,986 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 24, 985936), True, datetime.datetime(2022, 5, 5, 10, 19, 24, 985936), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:19:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:19:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:19:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:19:25,947 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:19:26,549 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:19:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:19:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:56.865082+00:00 (in 29.999537 seconds) gcworker stdout | 2022-05-05 10:19:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:56 UTC)" (scheduled at 2022-05-05 10:19:26.865082+00:00) gcworker stdout | 2022-05-05 10:19:26,866 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536366866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:19:26,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:19:26,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:19:26,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:56 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:19:28,059 [222] [DEBUG] [app] Starting request: urn:request:d1d10dc8-ad27-4548-95da-4142f9c5a20b (/api/v1/user/apptoken) gunicorn-web stdout | 2022-05-05 10:19:28,059 [222] [DEBUG] [app] User loader loading deferred user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:19:28,059 [222] [DEBUG] [auth.cookie] Loading user from cookie: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:19:28,060 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['105a42ac-0110-4aed-8579-1c532997244a', False, 1, 0]) gunicorn-web stdout | 2022-05-05 10:19:28,062 [219] [DEBUG] [app] Starting request: urn:request:f7657b71-dbbc-4fcf-aeea-b38a033cf6f0 (/api/v1/user/notifications) gunicorn-web stdout | 2022-05-05 10:19:28,063 [219] [DEBUG] [app] User loader loading deferred user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:19:28,063 [219] [DEBUG] [auth.cookie] Loading user from cookie: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:19:28,064 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."uuid", "t1"."username", "t1"."password_hash", "t1"."email", "t1"."verified", "t1"."stripe_id", "t1"."organization", "t1"."robot", "t1"."invoice_email", "t1"."invalid_login_attempts", "t1"."last_invalid_login", "t1"."removed_tag_expiration_s", "t1"."enabled", "t1"."invoice_email_address", "t1"."given_name", "t1"."family_name", "t1"."company", "t1"."location", "t1"."maximum_queued_builds_count", "t1"."creation_date", "t1"."last_accessed" FROM "user" AS "t1" WHERE (("t1"."uuid" = %s) AND ("t1"."organization" = %s)) LIMIT %s OFFSET %s', ['105a42ac-0110-4aed-8579-1c532997244a', False, 1, 0]) gunicorn-web stdout | 2022-05-05 10:19:28,070 [222] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-05 10:19:28,070 [222] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-05 10:19:28,070 [222] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:19:28,070 [222] [DEBUG] [endpoints.api] Checking permission for user ibazulic gunicorn-web stdout | 2022-05-05 10:19:28,071 [222] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:19:28,071 [222] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:19:28,071 [222] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:19:28,071 [222] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:19:28,072 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."user_id", "t1"."uuid", "t1"."title", "t1"."token_name", "t1"."token_secret", "t1"."created", "t1"."expiration", "t1"."last_accessed" FROM "appspecificauthtoken" AS "t1" WHERE ((("t1"."user_id" = %s) AND ("t1"."expiration" <= %s)) AND ("t1"."expiration" > %s))', [1, datetime.datetime(2022, 5, 8, 5, 31, 28, 71361), datetime.datetime(2022, 5, 5, 10, 19, 28, 71423)]) gunicorn-web stdout | 2022-05-05 10:19:28,072 [219] [DEBUG] [auth.decorators] Found valid auth result: (, False, None, [, None, None, None, None, None]) gunicorn-web stdout | 2022-05-05 10:19:28,072 [219] [DEBUG] [auth.permissions] Identity loaded: gunicorn-web stdout | 2022-05-05 10:19:28,072 [219] [DEBUG] [auth.permissions] Deferring permissions for user with uuid: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:19:28,072 [219] [DEBUG] [endpoints.api] Checking permission for user ibazulic gunicorn-web stdout | 2022-05-05 10:19:28,073 [219] [DEBUG] [auth.permissions] Loading user permissions after deferring for: 105a42ac-0110-4aed-8579-1c532997244a gunicorn-web stdout | 2022-05-05 10:19:28,073 [219] [DEBUG] [auth.permissions] User permission: userspecificneed(type='user', username='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:19:28,073 [219] [DEBUG] [auth.permissions] User namespace permission: namespacewide(type='organization', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:19:28,073 [219] [DEBUG] [auth.permissions] User namespace repo permission: namespacewide(type='organizationrepo', namespace='ibazulic', role='admin') gunicorn-web stdout | 2022-05-05 10:19:28,075 [222] [DEBUG] [app] Ending request: urn:request:d1d10dc8-ad27-4548-95da-4142f9c5a20b (/api/v1/user/apptoken) gunicorn-web stdout | 2022-05-05 10:19:28,076 [219] [DEBUG] [peewee] ('(SELECT "t1"."id", "t1"."uuid", "t1"."kind_id", "t1"."metadata_json", "t1"."dismissed", "t1"."lookup_path", "t1"."created", "t1"."created" AS "cd", "t1"."target_id" FROM "notification" AS "t1" INNER JOIN "notificationkind" AS "t2" ON ("t1"."kind_id" = "t2"."id") WHERE (("t1"."dismissed" = %s) AND ("t1"."target_id" = %s))) UNION (SELECT "t3"."id", "t3"."uuid", "t3"."kind_id", "t3"."metadata_json", "t3"."dismissed", "t3"."lookup_path", "t3"."created", "t3"."created" AS "cd", "t3"."target_id" FROM "notification" AS "t3" INNER JOIN "notificationkind" AS "t4" ON ("t3"."kind_id" = "t4"."id") INNER JOIN "user" AS "t5" ON ("t5"."id" = "t3"."target_id") INNER JOIN "team" AS "t6" ON ("t5"."id" = "t6"."organization_id") INNER JOIN "teamrole" AS "t7" ON ("t6"."role_id" = "t7"."id") INNER JOIN "teammember" AS "t8" ON ("t6"."id" = "t8"."team_id") INNER JOIN "user" AS "t9" ON ("t8"."user_id" = "t9"."id") WHERE (("t3"."dismissed" = %s) AND (("t9"."id" = %s) AND ("t7"."name" = %s)))) ORDER BY cd desc LIMIT %s', [False, 1, False, 1, 'admin', 6]) gunicorn-web stdout | 2022-05-05 10:19:28,076 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:28,077 [222] [INFO] [gunicorn.access] 10.130.2.11 - - [05/May/2022:10:19:28 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.0" 200 38 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" nginx stdout | 10.130.2.11 () - - [05/May/2022:10:19:28 +0000] "GET /api/v1/user/apptoken?expiring=true HTTP/1.1" 200 38 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" (0.019 1303 0.019) gunicorn-web stdout | 2022-05-05 10:19:28,081 [219] [DEBUG] [app] Ending request: urn:request:f7657b71-dbbc-4fcf-aeea-b38a033cf6f0 (/api/v1/user/notifications) gunicorn-web stdout | 2022-05-05 10:19:28,082 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:28,083 [219] [INFO] [gunicorn.access] 10.128.2.6 - - [05/May/2022:10:19:28 +0000] "GET /api/v1/user/notifications HTTP/1.0" 200 43 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" nginx stdout | 10.128.2.6 () - - [05/May/2022:10:19:28 +0000] "GET /api/v1/user/notifications HTTP/1.1" 200 43 "https://quay-quay-quay-enterprise.apps.rcegan.azemea.com/repository/ibazulic/minio?tab=tags" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0" (0.022 1294 0.021) securityworker stdout | 2022-05-05 10:19:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:19:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:58.201650+00:00 (in 29.999534 seconds) securityworker stdout | 2022-05-05 10:19:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:58 UTC)" (scheduled at 2022-05-05 10:19:28.201650+00:00) securityworker stdout | 2022-05-05 10:19:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:19:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:19:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:19:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:19:28,215 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:19:28,218 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:19:28,220 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:19:28,220 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:19:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 14, 28, 220984), 1, 2]) securityworker stdout | 2022-05-05 10:19:28,223 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:19:28,223 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:19:28,224 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 14, 28, 224082), 1, 2]) securityworker stdout | 2022-05-05 10:19:28,226 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:19:28,226 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:19:28,226 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:19:28,226 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:19:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:19:30,328 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:19:30,429 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:19:30,878 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} chunkcleanupworker stdout | 2022-05-05 10:19:31,759 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:19:31,770 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} securityscanningnotificationworker stdout | 2022-05-05 10:19:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:19:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:35.969483+00:00 (in 4.194645 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:19:31,775 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:31 UTC)" (scheduled at 2022-05-05 10:19:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:19:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:19:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 31, 775308), True, datetime.datetime(2022, 5, 5, 10, 19, 31, 775308), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:19:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:19:31,786 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:19:31,786 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:31 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:19:32,056 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:19:32,370 [220] [DEBUG] [app] Starting request: urn:request:d1d23171-a0ec-420a-bd72-803f68578096 (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:32,370 [221] [DEBUG] [app] Starting request: urn:request:4d645cc4-fadb-4858-adf3-987ba1d5d7bf (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:32,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:32,371 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:19:32,373 [292] [DEBUG] [app] Starting request: urn:request:db654671-a0f5-4d19-b55d-77bda207e038 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:32,373 [288] [DEBUG] [app] Starting request: urn:request:02ec7494-eb72-4e65-8229-5e9cfd167a93 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:32,374 [292] [DEBUG] [app] Ending request: urn:request:db654671-a0f5-4d19-b55d-77bda207e038 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:32,374 [288] [DEBUG] [app] Ending request: urn:request:02ec7494-eb72-4e65-8229-5e9cfd167a93 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:32,374 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:19:32,374 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:19:32,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:32,375 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:19:32,376 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:32,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:32,378 [222] [DEBUG] [app] Starting request: urn:request:e92c8843-6a50-4f43-86ec-0476ec49d4b2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:32,378 [220] [DEBUG] [app] Starting request: urn:request:3e642190-8f1f-44c2-a810-f75e11083f75 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:32,379 [222] [DEBUG] [app] Ending request: urn:request:e92c8843-6a50-4f43-86ec-0476ec49d4b2 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:32,379 [220] [DEBUG] [app] Ending request: urn:request:3e642190-8f1f-44c2-a810-f75e11083f75 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.001) gunicorn-web stdout | 2022-05-05 10:19:32,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:19:32,379 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:19:32,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:32,380 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:32,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:19:32,380 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:19:32,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:19:32,387 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:19:32,394 [221] [DEBUG] [app] Ending request: urn:request:4d645cc4-fadb-4858-adf3-987ba1d5d7bf (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:32,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:32,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:19:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:19:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:19:32,395 [220] [DEBUG] [app] Ending request: urn:request:d1d23171-a0ec-420a-bd72-803f68578096 (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:32,395 [220] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:19:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-web stdout | 2022-05-05 10:19:32,396 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:19:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" notificationworker stdout | 2022-05-05 10:19:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:19:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:43.089985+00:00 (in 9.999455 seconds) notificationworker stdout | 2022-05-05 10:19:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:43 UTC)" (scheduled at 2022-05-05 10:19:33.089985+00:00) notificationworker stdout | 2022-05-05 10:19:33,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:19:33,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 33, 91000), True, datetime.datetime(2022, 5, 5, 10, 19, 33, 91000), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:19:33,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:19:33,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:19:33,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:43 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:19:33,132 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:19:33,142 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:19:33,146 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:19:33,487 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:19:33,530 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:19:33,950 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:19:34,080 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:19:34,195 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:19:34,936 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityscanningnotificationworker stdout | 2022-05-05 10:19:35,970 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:19:35,971 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:31.774027+00:00 (in 55.803113 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:19:35,971 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:35 UTC)" (scheduled at 2022-05-05 10:19:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:19:35,971 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:19:35,971 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:35 UTC)" executed successfully securityworker stdout | 2022-05-05 10:19:35,977 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} servicekey stdout | 2022-05-05 10:19:36,141 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} gunicorn-web stdout | 2022-05-05 10:19:36,146 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:19:36,154 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:19:36,165 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:19:36,174 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:19:36,177 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} globalpromstats stdout | 2022-05-05 10:19:36,199 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:19:36,439 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:19:36,735 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:19:38,325 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:19:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:19:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:14.286471+00:00 (in 35.307998 seconds) namespacegcworker stdout | 2022-05-05 10:19:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:38 UTC)" (scheduled at 2022-05-05 10:19:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:19:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:19:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 38, 978975), True, datetime.datetime(2022, 5, 5, 10, 19, 38, 978975), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:19:38,990 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:19:38,990 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:19:38,990 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:19:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:19:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:19:53.089985+00:00 (in 9.999517 seconds) notificationworker stdout | 2022-05-05 10:19:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:53 UTC)" (scheduled at 2022-05-05 10:19:43.089985+00:00) notificationworker stdout | 2022-05-05 10:19:43,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:19:43,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 43, 90951), True, datetime.datetime(2022, 5, 5, 10, 19, 43, 90951), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:19:43,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:19:43,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:19:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:19:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:19:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:19:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:14.479974+00:00 (in 29.999458 seconds) buildlogsarchiver stdout | 2022-05-05 10:19:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:14 UTC)" (scheduled at 2022-05-05 10:19:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:19:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 19, 44, 481011), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:19:44,493 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:19:44,494 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:19:44,494 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:19:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:19:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:00.978909+00:00 (in 13.790941 seconds) exportactionlogsworker stdout | 2022-05-05 10:19:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:47 UTC)" (scheduled at 2022-05-05 10:19:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:19:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:19:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:20:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:19:47,369 [221] [DEBUG] [app] Starting request: urn:request:5f85ef17-ed91-44db-b263-35cf4d436f4b (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:47,369 [222] [DEBUG] [app] Starting request: urn:request:a48ca56f-6cc9-474a-b62e-7fe6209f398d (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:47,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:47,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:19:47,373 [292] [DEBUG] [app] Starting request: urn:request:a8832638-603d-466a-b7ed-e4a9134e6aa9 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:47,373 [288] [DEBUG] [app] Starting request: urn:request:39895996-cc22-4162-917f-5c8c6c9563ac (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:47,374 [292] [DEBUG] [app] Ending request: urn:request:a8832638-603d-466a-b7ed-e4a9134e6aa9 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:47,374 [288] [DEBUG] [app] Ending request: urn:request:39895996-cc22-4162-917f-5c8c6c9563ac (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:19:47,374 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:19:47,374 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:19:47,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:47,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:47,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:19:47,377 [221] [DEBUG] [app] Starting request: urn:request:8b6eb59e-dd47-4b18-bb42-483fceb6252e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:47,378 [221] [DEBUG] [app] Ending request: urn:request:8b6eb59e-dd47-4b18-bb42-483fceb6252e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:47,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:19:47,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:47,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:19:47,380 [219] [DEBUG] [app] Starting request: urn:request:87d92d4a-3329-4c4c-9fe0-cf5535920236 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:19:47,380 [219] [DEBUG] [app] Ending request: urn:request:87d92d4a-3329-4c4c-9fe0-cf5535920236 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:19:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:19:47,381 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:19:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:19:47,381 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:19:47,382 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:19:47,387 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:19:47,388 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:19:47,396 [221] [DEBUG] [app] Ending request: urn:request:5f85ef17-ed91-44db-b263-35cf4d436f4b (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:47,396 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:47,396 [222] [DEBUG] [app] Ending request: urn:request:a48ca56f-6cc9-474a-b62e-7fe6209f398d (/health/instance) gunicorn-web stdout | 2022-05-05 10:19:47,396 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:19:47,397 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:19:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-web stdout | 2022-05-05 10:19:47,397 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:19:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:19:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) nginx stdout | 10.130.2.1 () - - [05/May/2022:10:19:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-registry stdout | 2022-05-05 10:19:51,396 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:51,401 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:51,405 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:51,410 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:51,414 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:51,418 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:51,424 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:51,428 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:19:51,431 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:19:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:19:53,091 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:03.089985+00:00 (in 9.999255 seconds) notificationworker stdout | 2022-05-05 10:19:53,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:03 UTC)" (scheduled at 2022-05-05 10:19:53.089985+00:00) notificationworker stdout | 2022-05-05 10:19:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:19:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 19, 53, 91848), True, datetime.datetime(2022, 5, 5, 10, 19, 53, 91848), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:19:53,105 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:19:53,105 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:19:53,105 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:19:55,984 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:19:56,581 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:19:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:19:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:26.865082+00:00 (in 29.999565 seconds) gcworker stdout | 2022-05-05 10:19:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:26 UTC)" (scheduled at 2022-05-05 10:19:56.865082+00:00) gcworker stdout | 2022-05-05 10:19:56,866 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536396866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:19:56,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:19:56,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:19:56,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:19:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:19:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:28.201650+00:00 (in 29.999413 seconds) securityworker stdout | 2022-05-05 10:19:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:28 UTC)" (scheduled at 2022-05-05 10:19:58.201650+00:00) securityworker stdout | 2022-05-05 10:19:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:19:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:19:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:19:58,207 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:19:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:19:58,216 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:28 UTC)" executed successfully gcworker stdout | 2022-05-05 10:20:00,341 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:20:00,461 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:20:00,914 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:20:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:20:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:47.187465+00:00 (in 46.208103 seconds) exportactionlogsworker stdout | 2022-05-05 10:20:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:00 UTC)" (scheduled at 2022-05-05 10:20:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:20:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:20:00,980 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 0, 979868), True, datetime.datetime(2022, 5, 5, 10, 20, 0, 979868), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:20:00,992 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:20:00,992 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:20:00,992 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:20:01,777 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:20:01,804 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:20:02,090 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:20:02,370 [219] [DEBUG] [app] Starting request: urn:request:e3da4c8d-d6bc-415e-a807-b55b35665411 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:02,370 [221] [DEBUG] [app] Starting request: urn:request:924453f2-9b7f-40c7-a7d7-766593aac282 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:02,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:02,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:20:02,373 [291] [DEBUG] [app] Starting request: urn:request:8bbcd9cf-17b1-4e5f-93d1-009f46309a97 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:02,373 [288] [DEBUG] [app] Starting request: urn:request:47d353db-e0ab-47ad-bbca-e31ce61cfb1a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:02,374 [288] [DEBUG] [app] Ending request: urn:request:47d353db-e0ab-47ad-bbca-e31ce61cfb1a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:02,374 [291] [DEBUG] [app] Ending request: urn:request:8bbcd9cf-17b1-4e5f-93d1-009f46309a97 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:02,374 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:20:02,374 [291] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:20:02,374 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:02,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) gunicorn-web stdout | 2022-05-05 10:20:02,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:02,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:02,378 [221] [DEBUG] [app] Starting request: urn:request:3ed3d41a-a8d8-4324-b311-791f4417e71e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:02,378 [222] [DEBUG] [app] Starting request: urn:request:ff2f7103-f0cc-4e41-a209-76dba8570b41 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:02,378 [221] [DEBUG] [app] Ending request: urn:request:3ed3d41a-a8d8-4324-b311-791f4417e71e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:02,378 [222] [DEBUG] [app] Ending request: urn:request:ff2f7103-f0cc-4e41-a209-76dba8570b41 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:02,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:20:02,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:20:02,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:20:02,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:02,379 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:20:02,379 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:20:02,386 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:20:02,387 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:20:02,394 [221] [DEBUG] [app] Ending request: urn:request:924453f2-9b7f-40c7-a7d7-766593aac282 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:02,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:20:02,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:20:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:20:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:20:02,395 [219] [DEBUG] [app] Ending request: urn:request:e3da4c8d-d6bc-415e-a807-b55b35665411 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:02,396 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:20:02,396 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:20:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:20:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) notificationworker stdout | 2022-05-05 10:20:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:20:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:12.207990+00:00 (in 9.117224 seconds) notificationworker stdout | 2022-05-05 10:20:03,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:13 UTC)" (scheduled at 2022-05-05 10:20:03.089985+00:00) notificationworker stdout | 2022-05-05 10:20:03,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:20:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 3, 91237), True, datetime.datetime(2022, 5, 5, 10, 20, 3, 91237), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:20:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:20:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:20:03,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:13 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:20:03,142 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:20:03,150 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:20:03,153 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:20:03,509 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:20:03,558 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:20:03,986 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:20:04,104 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:20:04,232 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:20:04,965 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:20:06,011 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} gunicorn-web stdout | 2022-05-05 10:20:06,155 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:20:06,163 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:20:06,172 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} servicekey stdout | 2022-05-05 10:20:06,177 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} gunicorn-web stdout | 2022-05-05 10:20:06,183 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:20:06,186 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} globalpromstats stdout | 2022-05-05 10:20:06,216 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:20:06,471 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:20:06,766 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:20:08,362 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:20:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:20:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:13.089985+00:00 (in 0.881522 seconds) notificationworker stdout | 2022-05-05 10:20:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:12 UTC)" (scheduled at 2022-05-05 10:20:12.207990+00:00) notificationworker stdout | 2022-05-05 10:20:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:20:12,208 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:20:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:20:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:23.089985+00:00 (in 9.999497 seconds) notificationworker stdout | 2022-05-05 10:20:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:23 UTC)" (scheduled at 2022-05-05 10:20:13.089985+00:00) notificationworker stdout | 2022-05-05 10:20:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:20:13,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 13, 90962), True, datetime.datetime(2022, 5, 5, 10, 20, 13, 90962), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:20:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:20:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:20:13,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:20:13,394 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:20:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:24.984990+00:00 (in 11.589933 seconds) repositorygcworker stdout | 2022-05-05 10:20:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:13 UTC)" (scheduled at 2022-05-05 10:20:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:20:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:20:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:20:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:20:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:38.977973+00:00 (in 24.691008 seconds) namespacegcworker stdout | 2022-05-05 10:20:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:14 UTC)" (scheduled at 2022-05-05 10:20:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:20:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:20:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:20:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:20:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:44.479974+00:00 (in 29.999508 seconds) buildlogsarchiver stdout | 2022-05-05 10:20:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:44 UTC)" (scheduled at 2022-05-05 10:20:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:20:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 20, 14, 480951), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:20:14,492 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:20:14,492 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:20:14,492 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:20:17,370 [219] [DEBUG] [app] Starting request: urn:request:35cdffa3-9465-478b-b7b7-a4e065164fc9 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:17,370 [220] [DEBUG] [app] Starting request: urn:request:727f38e5-2ef7-4691-ad90-c9e640063d3c (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:17,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:17,372 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:20:17,374 [292] [DEBUG] [app] Starting request: urn:request:03c929d9-a96d-4ea1-8405-a722266f5b10 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:17,374 [288] [DEBUG] [app] Starting request: urn:request:2c5cfaa1-4252-4de0-9814-6ad2c5f2bf1a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:17,375 [288] [DEBUG] [app] Ending request: urn:request:2c5cfaa1-4252-4de0-9814-6ad2c5f2bf1a (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:17,375 [292] [DEBUG] [app] Ending request: urn:request:03c929d9-a96d-4ea1-8405-a722266f5b10 (/v1/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.003 162 0.002) gunicorn-registry stdout | 2022-05-05 10:20:17,376 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:20:17,376 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:20:17,376 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:17,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:17,378 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:17,378 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:17,380 [222] [DEBUG] [app] Starting request: urn:request:f4a5e3fb-ca46-4b4a-ab22-bad6dfc6ecc4 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:17,380 [221] [DEBUG] [app] Starting request: urn:request:2892a9d4-0552-4bf1-82b3-e8827f5cc80e (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:17,381 [222] [DEBUG] [app] Ending request: urn:request:f4a5e3fb-ca46-4b4a-ab22-bad6dfc6ecc4 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:17,381 [221] [DEBUG] [app] Ending request: urn:request:2892a9d4-0552-4bf1-82b3-e8827f5cc80e (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:20:17,381 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:20:17,381 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:20:17,382 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:17,382 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:17,382 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:20:17,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:20:17,389 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:20:17,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:20:17,396 [220] [DEBUG] [app] Ending request: urn:request:727f38e5-2ef7-4691-ad90-c9e640063d3c (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:17,396 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:20:17,396 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:20:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:20:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) gunicorn-web stdout | 2022-05-05 10:20:17,397 [219] [DEBUG] [app] Ending request: urn:request:35cdffa3-9465-478b-b7b7-a4e065164fc9 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:17,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:20:17,397 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:20:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:20:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-registry stdout | 2022-05-05 10:20:21,407 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:21,411 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:21,416 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:21,421 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:21,424 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:21,428 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:21,438 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:21,442 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:21,446 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:20:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:20:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:32.207092+00:00 (in 9.116636 seconds) notificationworker stdout | 2022-05-05 10:20:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:33 UTC)" (scheduled at 2022-05-05 10:20:23.089985+00:00) notificationworker stdout | 2022-05-05 10:20:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:20:23,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 23, 90936), True, datetime.datetime(2022, 5, 5, 10, 20, 23, 90936), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:20:23,100 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:20:23,100 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:20:23,100 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:20:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:20:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:13.394619+00:00 (in 48.409158 seconds) repositorygcworker stdout | 2022-05-05 10:20:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:24 UTC)" (scheduled at 2022-05-05 10:20:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:20:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:20:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 24, 986004), True, datetime.datetime(2022, 5, 5, 10, 20, 24, 986004), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:20:24,997 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:20:24,997 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:20:24,997 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:20:26,011 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:20:26,598 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:20:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:20:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:56.865082+00:00 (in 29.999513 seconds) gcworker stdout | 2022-05-05 10:20:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:56 UTC)" (scheduled at 2022-05-05 10:20:26.865082+00:00) gcworker stdout | 2022-05-05 10:20:26,866 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536426866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:20:26,878 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:20:26,878 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:20:26,878 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:20:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:20:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:58.201650+00:00 (in 29.999445 seconds) securityworker stdout | 2022-05-05 10:20:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:58 UTC)" (scheduled at 2022-05-05 10:20:28.201650+00:00) securityworker stdout | 2022-05-05 10:20:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:20:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:20:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:20:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:20:28,216 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:20:28,218 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:20:28,220 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:20:28,220 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:20:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 15, 28, 220491), 1, 2]) securityworker stdout | 2022-05-05 10:20:28,222 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:20:28,222 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:20:28,223 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 15, 28, 222701), 1, 2]) securityworker stdout | 2022-05-05 10:20:28,225 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:20:28,225 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:20:28,225 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:20:28,225 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:20:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:20:30,374 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:20:30,495 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:20:30,948 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} securityscanningnotificationworker stdout | 2022-05-05 10:20:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:20:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:35.969483+00:00 (in 4.194957 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:20:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:31 UTC)" (scheduled at 2022-05-05 10:20:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:20:31,775 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:20:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 31, 775088), True, datetime.datetime(2022, 5, 5, 10, 20, 31, 775088), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:20:31,787 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:20:31,787 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:20:31,787 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:31 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:20:31,814 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:20:31,840 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:20:32,127 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} notificationworker stdout | 2022-05-05 10:20:32,207 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:20:32,207 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:33.089985+00:00 (in 0.882406 seconds) notificationworker stdout | 2022-05-05 10:20:32,207 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:25:32 UTC)" (scheduled at 2022-05-05 10:20:32.207092+00:00) notificationworker stdout | 2022-05-05 10:20:32,208 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 20, 32, 208034), 'notification/%']) notificationworker stdout | 2022-05-05 10:20:32,217 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 20, 32, 208034), True, datetime.datetime(2022, 5, 5, 10, 20, 32, 208034), 0, 'notification/%']) notificationworker stdout | 2022-05-05 10:20:32,219 [76] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 20, 32, 208034), True, datetime.datetime(2022, 5, 5, 10, 20, 32, 208034), 0, 'notification/%', False, datetime.datetime(2022, 5, 5, 10, 20, 32, 208034), 'notification/%']) notificationworker stdout | 2022-05-05 10:20:32,222 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:20:32,222 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:25:32 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:20:32,370 [219] [DEBUG] [app] Starting request: urn:request:35d2c7a8-d11f-49c2-8a6c-9702f3648f98 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:32,370 [220] [DEBUG] [app] Starting request: urn:request:c5139798-ee5f-4135-90ec-14933ee7ba89 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:32,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:32,372 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:20:32,374 [291] [DEBUG] [app] Starting request: urn:request:0c6e34bc-f307-49fd-bc55-ab8f644d59fb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:32,374 [288] [DEBUG] [app] Starting request: urn:request:73804804-9d98-4d67-8e7e-4fe9dc8f5325 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:32,375 [288] [DEBUG] [app] Ending request: urn:request:73804804-9d98-4d67-8e7e-4fe9dc8f5325 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:32,375 [291] [DEBUG] [app] Ending request: urn:request:0c6e34bc-f307-49fd-bc55-ab8f644d59fb (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:32,375 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:20:32,375 [291] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:20:32,375 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-web stdout | 2022-05-05 10:20:32,376 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:32,378 [220] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:32,378 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:32,380 [222] [DEBUG] [app] Starting request: urn:request:769f206a-1f74-43ee-bca3-15ccda296ac8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:32,380 [220] [DEBUG] [app] Starting request: urn:request:752c58e5-4bb4-42d1-b746-9941bb0cb6b6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:32,380 [222] [DEBUG] [app] Ending request: urn:request:769f206a-1f74-43ee-bca3-15ccda296ac8 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:32,381 [220] [DEBUG] [app] Ending request: urn:request:752c58e5-4bb4-42d1-b746-9941bb0cb6b6 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:32,381 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:20:32,381 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:20:32,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:32,381 [220] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:32,382 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:20:32,382 [220] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:20:32,389 [220] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:20:32,390 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:20:32,396 [220] [DEBUG] [app] Ending request: urn:request:c5139798-ee5f-4135-90ec-14933ee7ba89 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:32,396 [220] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:20:32,397 [220] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:20:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:20:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.028 118 0.028) gunicorn-web stdout | 2022-05-05 10:20:32,397 [219] [DEBUG] [app] Ending request: urn:request:35d2c7a8-d11f-49c2-8a6c-9702f3648f98 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:32,397 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:20:32,398 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:20:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:20:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.029 118 0.029) notificationworker stdout | 2022-05-05 10:20:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:20:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:43.089985+00:00 (in 9.999350 seconds) notificationworker stdout | 2022-05-05 10:20:33,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:43 UTC)" (scheduled at 2022-05-05 10:20:33.089985+00:00) notificationworker stdout | 2022-05-05 10:20:33,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:20:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 33, 91205), True, datetime.datetime(2022, 5, 5, 10, 20, 33, 91205), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:20:33,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:20:33,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:20:33,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:43 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:20:33,150 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:20:33,158 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:20:33,162 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:20:33,546 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:20:33,595 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:20:34,014 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:20:34,129 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:20:34,262 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:20:34,988 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityscanningnotificationworker stdout | 2022-05-05 10:20:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:20:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:31.774027+00:00 (in 55.803973 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:20:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:35 UTC)" (scheduled at 2022-05-05 10:20:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:20:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:20:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:35 UTC)" executed successfully securityworker stdout | 2022-05-05 10:20:36,028 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} gunicorn-web stdout | 2022-05-05 10:20:36,163 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:20:36,171 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:20:36,181 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:20:36,190 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:20:36,194 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} servicekey stdout | 2022-05-05 10:20:36,212 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:20:36,238 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:20:36,504 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:20:36,801 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:20:38,377 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:20:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:20:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:14.286471+00:00 (in 35.307978 seconds) namespacegcworker stdout | 2022-05-05 10:20:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:38 UTC)" (scheduled at 2022-05-05 10:20:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:20:38,979 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:20:38,980 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 38, 979040), True, datetime.datetime(2022, 5, 5, 10, 20, 38, 979040), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:20:38,990 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:20:38,990 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:20:38,990 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:20:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:20:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:53.089985+00:00 (in 9.999243 seconds) notificationworker stdout | 2022-05-05 10:20:43,091 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:53 UTC)" (scheduled at 2022-05-05 10:20:43.089985+00:00) notificationworker stdout | 2022-05-05 10:20:43,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:20:43,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 43, 91310), True, datetime.datetime(2022, 5, 5, 10, 20, 43, 91310), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:20:43,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:20:43,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:20:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:20:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:20:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:20:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:14.479974+00:00 (in 29.999495 seconds) buildlogsarchiver stdout | 2022-05-05 10:20:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:14 UTC)" (scheduled at 2022-05-05 10:20:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:20:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 20, 44, 480959), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:20:44,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:20:44,490 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:20:44,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:20:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:20:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:20:59.184038+00:00 (in 11.996072 seconds) exportactionlogsworker stdout | 2022-05-05 10:20:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:47 UTC)" (scheduled at 2022-05-05 10:20:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:20:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:20:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:21:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:20:47,370 [221] [DEBUG] [app] Starting request: urn:request:23aa58e5-0eb9-4735-bd77-0d8b1eccfc58 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:47,370 [219] [DEBUG] [app] Starting request: urn:request:de5eabfe-aec1-4116-ad8f-3bcc77605eeb (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:47,371 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:47,372 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:20:47,373 [288] [DEBUG] [app] Starting request: urn:request:b3ad1c3e-9bf7-4f80-a869-94f652333b74 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:47,373 [291] [DEBUG] [app] Starting request: urn:request:9aaec9f7-6323-4ab2-a4ad-8ce1e586e444 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:47,374 [288] [DEBUG] [app] Ending request: urn:request:b3ad1c3e-9bf7-4f80-a869-94f652333b74 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:47,374 [291] [DEBUG] [app] Ending request: urn:request:9aaec9f7-6323-4ab2-a4ad-8ce1e586e444 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:20:47,374 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:20:47,374 [291] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:20:47,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:47,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:47,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:20:47,378 [220] [DEBUG] [app] Starting request: urn:request:5a22b42b-4f6f-473b-a984-37f61ac2764a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:47,378 [222] [DEBUG] [app] Starting request: urn:request:14c51025-7ec3-4e13-ba05-54230bc54b39 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:47,379 [220] [DEBUG] [app] Ending request: urn:request:5a22b42b-4f6f-473b-a984-37f61ac2764a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:47,379 [222] [DEBUG] [app] Ending request: urn:request:14c51025-7ec3-4e13-ba05-54230bc54b39 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:20:47,379 [220] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:20:47,379 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:20:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:20:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:20:47,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:47,380 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:20:47,381 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:20:47,381 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:20:47,386 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:20:47,387 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:20:47,394 [221] [DEBUG] [app] Ending request: urn:request:23aa58e5-0eb9-4735-bd77-0d8b1eccfc58 (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:47,394 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:20:47,394 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:20:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:20:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:20:47,394 [219] [DEBUG] [app] Ending request: urn:request:de5eabfe-aec1-4116-ad8f-3bcc77605eeb (/health/instance) gunicorn-web stdout | 2022-05-05 10:20:47,395 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:20:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.026) gunicorn-web stdout | 2022-05-05 10:20:47,395 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:20:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:20:51,419 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:51,422 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:51,428 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:51,432 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:51,436 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:51,439 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:51,450 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:51,453 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:20:51,457 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:20:53,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:20:53,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:03.089985+00:00 (in 9.999411 seconds) notificationworker stdout | 2022-05-05 10:20:53,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:03 UTC)" (scheduled at 2022-05-05 10:20:53.089985+00:00) notificationworker stdout | 2022-05-05 10:20:53,091 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:20:53,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 20, 53, 91068), True, datetime.datetime(2022, 5, 5, 10, 20, 53, 91068), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:20:53,102 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:20:53,102 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:20:53,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:03 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:20:56,023 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:20:56,630 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:20:56,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:20:56,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:26.865082+00:00 (in 29.999490 seconds) gcworker stdout | 2022-05-05 10:20:56,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:26 UTC)" (scheduled at 2022-05-05 10:20:56.865082+00:00) gcworker stdout | 2022-05-05 10:20:56,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536456866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:20:56,879 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:20:56,879 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:20:56,879 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:26 UTC)" executed successfully securityworker stdout | 2022-05-05 10:20:58,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:20:58,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:28.201650+00:00 (in 29.999526 seconds) securityworker stdout | 2022-05-05 10:20:58,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:28 UTC)" (scheduled at 2022-05-05 10:20:58.201650+00:00) securityworker stdout | 2022-05-05 10:20:58,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:20:58,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:20:58,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:20:58,208 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:20:58,216 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:20:58,217 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:28 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:20:59,184 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:20:59,184 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:00.978909+00:00 (in 1.794391 seconds) exportactionlogsworker stdout | 2022-05-05 10:20:59,184 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:25:59 UTC)" (scheduled at 2022-05-05 10:20:59.184038+00:00) exportactionlogsworker stdout | 2022-05-05 10:20:59,185 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2022, 5, 5, 10, 20, 59, 184903), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:20:59,195 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 20, 59, 184903), True, datetime.datetime(2022, 5, 5, 10, 20, 59, 184903), 0, 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:20:59,198 [61] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2022, 5, 5, 10, 20, 59, 184903), True, datetime.datetime(2022, 5, 5, 10, 20, 59, 184903), 0, 'exportactionlogs/%', False, datetime.datetime(2022, 5, 5, 10, 20, 59, 184903), 'exportactionlogs/%']) exportactionlogsworker stdout | 2022-05-05 10:20:59,201 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:20:59,201 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2022-05-05 10:25:59 UTC)" executed successfully gcworker stdout | 2022-05-05 10:21:00,393 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:21:00,504 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:21:00,961 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} exportactionlogsworker stdout | 2022-05-05 10:21:00,979 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:21:00,979 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:47.187465+00:00 (in 46.208075 seconds) exportactionlogsworker stdout | 2022-05-05 10:21:00,979 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:00 UTC)" (scheduled at 2022-05-05 10:21:00.978909+00:00) exportactionlogsworker stdout | 2022-05-05 10:21:00,979 [61] [DEBUG] [workers.queueworker] Getting work item from queue. exportactionlogsworker stdout | 2022-05-05 10:21:00,980 [61] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 0, 979889), True, datetime.datetime(2022, 5, 5, 10, 21, 0, 979889), 0, 'exportactionlogs/%', 50, 1, 0]) exportactionlogsworker stdout | 2022-05-05 10:21:00,991 [61] [DEBUG] [workers.queueworker] No more work. exportactionlogsworker stdout | 2022-05-05 10:21:00,991 [61] [DEBUG] [data.database] Disconnecting from database. exportactionlogsworker stdout | 2022-05-05 10:21:00,991 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:00 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:21:01,851 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:21:01,876 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:21:02,144 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:21:02,369 [219] [DEBUG] [app] Starting request: urn:request:c082e544-1538-4734-9dd3-b184b1dbd786 (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:02,369 [222] [DEBUG] [app] Starting request: urn:request:14eb3515-cbe5-4b07-ab12-63f20d220706 (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:02,370 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:02,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:21:02,372 [289] [DEBUG] [app] Starting request: urn:request:e2754541-d79e-4df3-a3c5-c65c671cdc6d (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:02,372 [288] [DEBUG] [app] Starting request: urn:request:711bb66d-97f3-4e81-8301-343fd64d0ea3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:02,373 [288] [DEBUG] [app] Ending request: urn:request:711bb66d-97f3-4e81-8301-343fd64d0ea3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:02,373 [289] [DEBUG] [app] Ending request: urn:request:e2754541-d79e-4df3-a3c5-c65c671cdc6d (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:02,373 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:21:02,374 [289] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:02 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:21:02,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:02,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:02,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:02,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:02,377 [221] [DEBUG] [app] Starting request: urn:request:1330feb0-c6df-4a66-be25-219a3157f5c5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:02,377 [219] [DEBUG] [app] Starting request: urn:request:74bf7a73-f442-4e6e-9c9f-ddd46d5f27c4 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:02,378 [219] [DEBUG] [app] Ending request: urn:request:74bf7a73-f442-4e6e-9c9f-ddd46d5f27c4 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:02,378 [221] [DEBUG] [app] Ending request: urn:request:1330feb0-c6df-4a66-be25-219a3157f5c5 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:02,378 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:02,378 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:02,378 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:02 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:02,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:02 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:21:02,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:21:02,379 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:21:02,386 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:21:02,386 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:21:02,393 [222] [DEBUG] [app] Ending request: urn:request:14eb3515-cbe5-4b07-ab12-63f20d220706 (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:02,393 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:21:02,393 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:21:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:21:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:21:02,397 [219] [DEBUG] [app] Ending request: urn:request:c082e544-1538-4734-9dd3-b184b1dbd786 (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:02,397 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:21:02 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.030 118 0.030) gunicorn-web stdout | 2022-05-05 10:21:02,398 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:21:02 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" notificationworker stdout | 2022-05-05 10:21:03,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:21:03,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:12.207990+00:00 (in 9.117491 seconds) notificationworker stdout | 2022-05-05 10:21:03,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:13 UTC)" (scheduled at 2022-05-05 10:21:03.089985+00:00) notificationworker stdout | 2022-05-05 10:21:03,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:21:03,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 3, 90972), True, datetime.datetime(2022, 5, 5, 10, 21, 3, 90972), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:21:03,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:21:03,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:21:03,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:13 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:21:03,159 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:21:03,167 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:21:03,171 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:21:03,560 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:21:03,628 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:21:04,042 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:21:04,148 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:21:04,299 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:21:05,026 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityworker stdout | 2022-05-05 10:21:06,035 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} gunicorn-web stdout | 2022-05-05 10:21:06,171 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:21:06,179 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:21:06,188 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:21:06,198 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:21:06,201 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} servicekey stdout | 2022-05-05 10:21:06,219 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:21:06,270 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:21:06,518 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:21:06,833 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:21:08,414 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} notificationworker stdout | 2022-05-05 10:21:12,208 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:21:12,208 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:13.089985+00:00 (in 0.881512 seconds) notificationworker stdout | 2022-05-05 10:21:12,208 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:12 UTC)" (scheduled at 2022-05-05 10:21:12.207990+00:00) notificationworker stdout | 2022-05-05 10:21:12,208 [76] [DEBUG] [workers.queueworker] Running watchdog. notificationworker stdout | 2022-05-05 10:21:12,208 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:12 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:21:13,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:21:13,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:23.089985+00:00 (in 9.999490 seconds) notificationworker stdout | 2022-05-05 10:21:13,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:23 UTC)" (scheduled at 2022-05-05 10:21:13.089985+00:00) notificationworker stdout | 2022-05-05 10:21:13,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:21:13,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 13, 90922), True, datetime.datetime(2022, 5, 5, 10, 21, 13, 90922), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:21:13,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:21:13,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:21:13,101 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:23 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:21:13,395 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:21:13,395 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:24.984990+00:00 (in 11.589809 seconds) repositorygcworker stdout | 2022-05-05 10:21:13,395 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:13 UTC)" (scheduled at 2022-05-05 10:21:13.394619+00:00) repositorygcworker stdout | 2022-05-05 10:21:13,395 [93] [DEBUG] [workers.queueworker] Running watchdog. repositorygcworker stdout | 2022-05-05 10:21:13,395 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:13 UTC)" executed successfully namespacegcworker stdout | 2022-05-05 10:21:14,286 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:21:14,287 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:38.977973+00:00 (in 24.690927 seconds) namespacegcworker stdout | 2022-05-05 10:21:14,287 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:14 UTC)" (scheduled at 2022-05-05 10:21:14.286471+00:00) namespacegcworker stdout | 2022-05-05 10:21:14,287 [71] [DEBUG] [workers.queueworker] Running watchdog. namespacegcworker stdout | 2022-05-05 10:21:14,287 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:14 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:21:14,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:21:14,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:44.479974+00:00 (in 29.999503 seconds) buildlogsarchiver stdout | 2022-05-05 10:21:14,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:44 UTC)" (scheduled at 2022-05-05 10:21:14.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:21:14,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 21, 14, 480931), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:21:14,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:21:14,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:21:14,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:44 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:21:17,369 [219] [DEBUG] [app] Starting request: urn:request:45d4d37a-c7bf-4216-9cf8-f90ef7eaa1cd (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:17,369 [222] [DEBUG] [app] Starting request: urn:request:acb8fadb-1422-4dfc-bf5d-c89031de8e13 (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:17,371 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:17,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:21:17,372 [292] [DEBUG] [app] Starting request: urn:request:239d93e5-60de-4c09-8f03-80f6f8a4f582 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:17,372 [288] [DEBUG] [app] Starting request: urn:request:ed9f4980-85bc-4a88-8a0a-fbefeef56eed (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:17,373 [288] [DEBUG] [app] Ending request: urn:request:ed9f4980-85bc-4a88-8a0a-fbefeef56eed (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:17,373 [292] [DEBUG] [app] Ending request: urn:request:239d93e5-60de-4c09-8f03-80f6f8a4f582 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:17,373 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-registry stdout | 2022-05-05 10:21:17,373 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:17 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:21:17,374 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:17,374 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:17,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:17,376 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:17,377 [222] [DEBUG] [app] Starting request: urn:request:9d06b5dc-721a-45b0-928b-a67ab2739a0a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:17,377 [219] [DEBUG] [app] Starting request: urn:request:5c048a44-cbf7-414d-b500-57d3f2bbf7d9 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:17,378 [222] [DEBUG] [app] Ending request: urn:request:9d06b5dc-721a-45b0-928b-a67ab2739a0a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:17,378 [219] [DEBUG] [app] Ending request: urn:request:5c048a44-cbf7-414d-b500-57d3f2bbf7d9 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:17 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.001) gunicorn-web stdout | 2022-05-05 10:21:17,378 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:17,378 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:17 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:17,378 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:17,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:17,379 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:21:17,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:21:17,385 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:21:17,385 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:21:17,392 [222] [DEBUG] [app] Ending request: urn:request:acb8fadb-1422-4dfc-bf5d-c89031de8e13 (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:17,392 [222] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:21:17,393 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:21:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:21:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.024 118 0.025) gunicorn-web stdout | 2022-05-05 10:21:17,393 [219] [DEBUG] [app] Ending request: urn:request:45d4d37a-c7bf-4216-9cf8-f90ef7eaa1cd (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:17,393 [219] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:21:17 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.026) gunicorn-web stdout | 2022-05-05 10:21:17,394 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:21:17 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" gunicorn-registry stdout | 2022-05-05 10:21:21,429 [290] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:21:21,432 [291] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:21:21,437 [64] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:21:21,443 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:21:21,447 [292] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:21:21,451 [288] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:21:21,461 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:21:21,464 [289] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} gunicorn-registry stdout | 2022-05-05 10:21:21,469 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'registry:application', 'pid': '64'} notificationworker stdout | 2022-05-05 10:21:23,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:21:23,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:33.089985+00:00 (in 9.999477 seconds) notificationworker stdout | 2022-05-05 10:21:23,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:33 UTC)" (scheduled at 2022-05-05 10:21:23.089985+00:00) notificationworker stdout | 2022-05-05 10:21:23,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:21:23,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 23, 90926), True, datetime.datetime(2022, 5, 5, 10, 21, 23, 90926), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:21:23,100 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:21:23,100 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:21:23,100 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:33 UTC)" executed successfully repositorygcworker stdout | 2022-05-05 10:21:24,985 [93] [DEBUG] [apscheduler.scheduler] Looking for jobs to run repositorygcworker stdout | 2022-05-05 10:21:24,985 [93] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:22:13.394619+00:00 (in 48.409166 seconds) repositorygcworker stdout | 2022-05-05 10:21:24,985 [93] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:24 UTC)" (scheduled at 2022-05-05 10:21:24.984990+00:00) repositorygcworker stdout | 2022-05-05 10:21:24,985 [93] [DEBUG] [workers.queueworker] Getting work item from queue. repositorygcworker stdout | 2022-05-05 10:21:24,987 [93] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 24, 985931), True, datetime.datetime(2022, 5, 5, 10, 21, 24, 985931), 0, 'repositorygc/%', 50, 1, 0]) repositorygcworker stdout | 2022-05-05 10:21:24,996 [93] [DEBUG] [workers.queueworker] No more work. repositorygcworker stdout | 2022-05-05 10:21:24,996 [93] [DEBUG] [data.database] Disconnecting from database. repositorygcworker stdout | 2022-05-05 10:21:24,996 [93] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:24 UTC)" executed successfully storagereplication stdout | 2022-05-05 10:21:26,031 [101] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'storagereplication.py', 'pid': '101'} expiredappspecifictokenworker stdout | 2022-05-05 10:21:26,661 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '60'} gcworker stdout | 2022-05-05 10:21:26,865 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run gcworker stdout | 2022-05-05 10:21:26,865 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:56.865082+00:00 (in 29.999504 seconds) gcworker stdout | 2022-05-05 10:21:26,865 [62] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:56 UTC)" (scheduled at 2022-05-05 10:21:26.865082+00:00) gcworker stdout | 2022-05-05 10:21:26,867 [62] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1650536486866, 1209600, True, 3, 500, 1, 0]) gcworker stdout | 2022-05-05 10:21:26,877 [62] [DEBUG] [__main__] No repository with garbage found gcworker stdout | 2022-05-05 10:21:26,877 [62] [DEBUG] [data.database] Disconnecting from database. gcworker stdout | 2022-05-05 10:21:26,877 [62] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:56 UTC)" executed successfully securityworker stdout | 2022-05-05 10:21:28,202 [97] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityworker stdout | 2022-05-05 10:21:28,202 [97] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:58.201650+00:00 (in 29.999362 seconds) securityworker stdout | 2022-05-05 10:21:28,202 [97] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:58 UTC)" (scheduled at 2022-05-05 10:21:28.201650+00:00) securityworker stdout | 2022-05-05 10:21:28,202 [97] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request securityworker stdout | 2022-05-05 10:21:28,202 [97] [DEBUG] [util.secscan.v4.api] GETing security URL http://quay-clair-app:80/indexer/api/v1/index_state securityworker stdout | 2022-05-05 10:21:28,206 [97] [DEBUG] [urllib3.connectionpool] http://quay-clair-app:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None securityworker stdout | 2022-05-05 10:21:28,207 [97] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:21:28,216 [97] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', []) securityworker stdout | 2022-05-05 10:21:28,218 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2]) securityworker stdout | 2022-05-05 10:21:28,220 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:21:28,221 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:21:28,221 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2022, 5, 5, 10, 16, 28, 221129), 1, 2]) securityworker stdout | 2022-05-05 10:21:28,223 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:21:28,223 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:21:28,224 [97] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((((("t2"."index_status" != %s) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, '7a5f5333aeda3d3d3c679da74d74cab5', datetime.datetime(2022, 5, 5, 10, 16, 28, 223585), 1, 2]) securityworker stdout | 2022-05-05 10:21:28,226 [97] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2 securityworker stdout | 2022-05-05 10:21:28,226 [97] [INFO] [util.migrate.allocator] No more work securityworker stdout | 2022-05-05 10:21:28,226 [97] [DEBUG] [data.database] Disconnecting from database. securityworker stdout | 2022-05-05 10:21:28,226 [97] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2022-05-05 10:21:58 UTC)" executed successfully gcworker stdout | 2022-05-05 10:21:30,430 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'gcworker.py', 'pid': '62'} blobuploadcleanupworker stdout | 2022-05-05 10:21:30,540 [55] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'blobuploadcleanupworker.py', 'pid': '55'} namespacegcworker stdout | 2022-05-05 10:21:30,970 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'namespacegcworker.py', 'pid': '71'} securityscanningnotificationworker stdout | 2022-05-05 10:21:31,774 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:21:31,774 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:35.969483+00:00 (in 4.194978 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:21:31,774 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:31 UTC)" (scheduled at 2022-05-05 10:21:31.774027+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:21:31,774 [94] [DEBUG] [workers.queueworker] Getting work item from queue. securityscanningnotificationworker stdout | 2022-05-05 10:21:31,776 [94] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 31, 774983), True, datetime.datetime(2022, 5, 5, 10, 21, 31, 774983), 0, 'secscanv4/%', 50, 1, 0]) securityscanningnotificationworker stdout | 2022-05-05 10:21:31,786 [94] [DEBUG] [workers.queueworker] No more work. securityscanningnotificationworker stdout | 2022-05-05 10:21:31,786 [94] [DEBUG] [data.database] Disconnecting from database. securityscanningnotificationworker stdout | 2022-05-05 10:21:31,786 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:31 UTC)" executed successfully chunkcleanupworker stdout | 2022-05-05 10:21:31,885 [58] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'chunkcleanupworker.py', 'pid': '58'} repositorygcworker stdout | 2022-05-05 10:21:31,890 [93] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositorygcworker.py', 'pid': '93'} exportactionlogsworker stdout | 2022-05-05 10:21:32,171 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'exportactionlogsworker.py', 'pid': '61'} gunicorn-web stdout | 2022-05-05 10:21:32,369 [219] [DEBUG] [app] Starting request: urn:request:b52f9c35-8ff4-418c-b2bd-e17c90c9bb9f (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:32,370 [221] [DEBUG] [app] Starting request: urn:request:9b1336e1-53dc-4053-9d83-5e75b5a892dd (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:32,371 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:32,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:21:32,373 [288] [DEBUG] [app] Starting request: urn:request:1772e73c-b52a-4022-a531-fc60889e359e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:32,374 [291] [DEBUG] [app] Starting request: urn:request:edf5ac37-4bc2-4790-aeaa-2faf510b0753 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:32,374 [288] [DEBUG] [app] Ending request: urn:request:1772e73c-b52a-4022-a531-fc60889e359e (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:32,374 [291] [DEBUG] [app] Ending request: urn:request:edf5ac37-4bc2-4790-aeaa-2faf510b0753 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:32,374 [288] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-web stdout | 2022-05-05 10:21:32,375 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 162 0.002) gunicorn-registry stdout | 2022-05-05 10:21:32,375 [291] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:32 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:32,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:32,377 [219] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:32,377 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:32,378 [221] [DEBUG] [app] Starting request: urn:request:100cc52e-fdad-4b41-b12f-80c223e3a676 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:32,379 [221] [DEBUG] [app] Ending request: urn:request:100cc52e-fdad-4b41-b12f-80c223e3a676 (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.001) gunicorn-web stdout | 2022-05-05 10:21:32,379 [221] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:32,379 [219] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:32,380 [219] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:21:32,381 [222] [DEBUG] [app] Starting request: urn:request:7e61d113-15dc-46ea-90de-7ef18b3ff79a (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:32,382 [222] [DEBUG] [app] Ending request: urn:request:7e61d113-15dc-46ea-90de-7ef18b3ff79a (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:32 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:21:32,382 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:32 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:32,382 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:32,383 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:21:32,386 [219] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:21:32,388 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:21:32,393 [219] [DEBUG] [app] Ending request: urn:request:b52f9c35-8ff4-418c-b2bd-e17c90c9bb9f (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:32,394 [219] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:21:32,394 [219] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:21:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:21:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.026 118 0.025) gunicorn-web stdout | 2022-05-05 10:21:32,395 [221] [DEBUG] [app] Ending request: urn:request:9b1336e1-53dc-4053-9d83-5e75b5a892dd (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:32,396 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:21:32,396 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:21:32 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:21:32 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.027 118 0.027) notificationworker stdout | 2022-05-05 10:21:33,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:21:33,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:43.089985+00:00 (in 9.999503 seconds) notificationworker stdout | 2022-05-05 10:21:33,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:43 UTC)" (scheduled at 2022-05-05 10:21:33.089985+00:00) notificationworker stdout | 2022-05-05 10:21:33,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:21:33,092 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 33, 90977), True, datetime.datetime(2022, 5, 5, 10, 21, 33, 90977), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:21:33,103 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:21:33,103 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:21:33,103 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:43 UTC)" executed successfully gunicorn-secscan stdout | 2022-05-05 10:21:33,168 [205] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '205'} gunicorn-secscan stdout | 2022-05-05 10:21:33,176 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '65'} gunicorn-secscan stdout | 2022-05-05 10:21:33,180 [204] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'secscan:application', 'pid': '204'} manifestbackfillworker stdout | 2022-05-05 10:21:33,597 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'manifestbackfillworker.py', 'pid': '69'} builder stdout | 2022-05-05 10:21:33,637 [56] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'builder.py', 'pid': '56'} buildlogsarchiver stdout | 2022-05-05 10:21:34,074 [57] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'buildlogsarchiver.py', 'pid': '57'} logrotateworker stdout | 2022-05-05 10:21:34,168 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'logrotateworker.py', 'pid': '68'} teamsyncworker stdout | 2022-05-05 10:21:34,314 [104] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'teamsyncworker.py', 'pid': '104'} repositoryactioncounter stdout | 2022-05-05 10:21:35,063 [92] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'repositoryactioncounter.py', 'pid': '92'} securityscanningnotificationworker stdout | 2022-05-05 10:21:35,969 [94] [DEBUG] [apscheduler.scheduler] Looking for jobs to run securityscanningnotificationworker stdout | 2022-05-05 10:21:35,970 [94] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:22:31.774027+00:00 (in 55.804027 seconds) securityscanningnotificationworker stdout | 2022-05-05 10:21:35,970 [94] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:35 UTC)" (scheduled at 2022-05-05 10:21:35.969483+00:00) securityscanningnotificationworker stdout | 2022-05-05 10:21:35,970 [94] [DEBUG] [workers.queueworker] Running watchdog. securityscanningnotificationworker stdout | 2022-05-05 10:21:35,970 [94] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:35 UTC)" executed successfully securityworker stdout | 2022-05-05 10:21:36,071 [97] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityworker.py', 'pid': '97'} gunicorn-web stdout | 2022-05-05 10:21:36,179 [220] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '220'} gunicorn-web stdout | 2022-05-05 10:21:36,187 [222] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '222'} gunicorn-web stdout | 2022-05-05 10:21:36,197 [221] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '221'} gunicorn-web stdout | 2022-05-05 10:21:36,205 [219] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '219'} gunicorn-web stdout | 2022-05-05 10:21:36,209 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'web:application', 'pid': '66'} servicekey stdout | 2022-05-05 10:21:36,256 [98] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'servicekeyworker.py', 'pid': '98'} globalpromstats stdout | 2022-05-05 10:21:36,301 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'globalpromstats.py', 'pid': '63'} notificationworker stdout | 2022-05-05 10:21:36,555 [76] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'notificationworker.py', 'pid': '76'} securityscanningnotificationworker stdout | 2022-05-05 10:21:36,870 [94] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'securityscanningnotificationworker.py', 'pid': '94'} queuecleanupworker stdout | 2022-05-05 10:21:38,451 [88] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quay-quay-app-d6556d95c-xg4lc', 'process_name': 'queuecleanupworker.py', 'pid': '88'} namespacegcworker stdout | 2022-05-05 10:21:38,978 [71] [DEBUG] [apscheduler.scheduler] Looking for jobs to run namespacegcworker stdout | 2022-05-05 10:21:38,978 [71] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:22:14.286471+00:00 (in 35.307932 seconds) namespacegcworker stdout | 2022-05-05 10:21:38,978 [71] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:38 UTC)" (scheduled at 2022-05-05 10:21:38.977973+00:00) namespacegcworker stdout | 2022-05-05 10:21:38,978 [71] [DEBUG] [workers.queueworker] Getting work item from queue. namespacegcworker stdout | 2022-05-05 10:21:38,979 [71] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 38, 979031), True, datetime.datetime(2022, 5, 5, 10, 21, 38, 979031), 0, 'namespacegc/%', 50, 1, 0]) namespacegcworker stdout | 2022-05-05 10:21:38,992 [71] [DEBUG] [workers.queueworker] No more work. namespacegcworker stdout | 2022-05-05 10:21:38,992 [71] [DEBUG] [data.database] Disconnecting from database. namespacegcworker stdout | 2022-05-05 10:21:38,992 [71] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:38 UTC)" executed successfully notificationworker stdout | 2022-05-05 10:21:43,090 [76] [DEBUG] [apscheduler.scheduler] Looking for jobs to run notificationworker stdout | 2022-05-05 10:21:43,090 [76] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:21:53.089985+00:00 (in 9.999457 seconds) notificationworker stdout | 2022-05-05 10:21:43,090 [76] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:53 UTC)" (scheduled at 2022-05-05 10:21:43.089985+00:00) notificationworker stdout | 2022-05-05 10:21:43,090 [76] [DEBUG] [workers.queueworker] Getting work item from queue. notificationworker stdout | 2022-05-05 10:21:43,091 [76] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2022, 5, 5, 10, 21, 43, 91023), True, datetime.datetime(2022, 5, 5, 10, 21, 43, 91023), 0, 'notification/%', 50, 1, 0]) notificationworker stdout | 2022-05-05 10:21:43,101 [76] [DEBUG] [workers.queueworker] No more work. notificationworker stdout | 2022-05-05 10:21:43,101 [76] [DEBUG] [data.database] Disconnecting from database. notificationworker stdout | 2022-05-05 10:21:43,102 [76] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2022-05-05 10:21:53 UTC)" executed successfully buildlogsarchiver stdout | 2022-05-05 10:21:44,480 [57] [DEBUG] [apscheduler.scheduler] Looking for jobs to run buildlogsarchiver stdout | 2022-05-05 10:21:44,480 [57] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:22:14.479974+00:00 (in 29.999499 seconds) buildlogsarchiver stdout | 2022-05-05 10:21:44,480 [57] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:22:14 UTC)" (scheduled at 2022-05-05 10:21:44.479974+00:00) buildlogsarchiver stdout | 2022-05-05 10:21:44,481 [57] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2022, 4, 20, 10, 21, 44, 480947), False, 50, 1, 0]) buildlogsarchiver stdout | 2022-05-05 10:21:44,490 [57] [DEBUG] [__main__] No more builds to archive buildlogsarchiver stdout | 2022-05-05 10:21:44,491 [57] [DEBUG] [data.database] Disconnecting from database. buildlogsarchiver stdout | 2022-05-05 10:21:44,491 [57] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2022-05-05 10:22:14 UTC)" executed successfully exportactionlogsworker stdout | 2022-05-05 10:21:47,187 [61] [DEBUG] [apscheduler.scheduler] Looking for jobs to run exportactionlogsworker stdout | 2022-05-05 10:21:47,188 [61] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2022-05-05 10:22:00.978909+00:00 (in 13.790971 seconds) exportactionlogsworker stdout | 2022-05-05 10:21:47,188 [61] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:47 UTC)" (scheduled at 2022-05-05 10:21:47.187465+00:00) exportactionlogsworker stdout | 2022-05-05 10:21:47,188 [61] [DEBUG] [workers.queueworker] Running watchdog. exportactionlogsworker stdout | 2022-05-05 10:21:47,188 [61] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2022-05-05 10:22:47 UTC)" executed successfully gunicorn-web stdout | 2022-05-05 10:21:47,369 [222] [DEBUG] [app] Starting request: urn:request:64feabe6-cf70-4e90-8904-86738f9c8e1c (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:47,369 [221] [DEBUG] [app] Starting request: urn:request:52f6d07f-db9e-4f62-b440-adeee38543be (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:47,372 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:47,372 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-registry stdout | 2022-05-05 10:21:47,374 [290] [DEBUG] [app] Starting request: urn:request:292edaab-07dd-43d4-a487-6c9e9ee9555f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:47,374 [292] [DEBUG] [app] Starting request: urn:request:c9389f2a-a1ec-4229-a663-42880a3e7df3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:47,374 [292] [DEBUG] [app] Ending request: urn:request:c9389f2a-a1ec-4229-a663-42880a3e7df3 (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:47,374 [290] [DEBUG] [app] Ending request: urn:request:292edaab-07dd-43d4-a487-6c9e9ee9555f (/v1/_internal_ping) gunicorn-registry stdout | 2022-05-05 10:21:47,375 [290] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 162 0.002) gunicorn-registry stdout | 2022-05-05 10:21:47,375 [292] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:47 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:47,375 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:47,375 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:47,376 [222] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:47,376 [221] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost gunicorn-web stdout | 2022-05-05 10:21:47,378 [222] [DEBUG] [app] Starting request: urn:request:029c6292-fc5f-4371-9ceb-65caa8157e23 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:47,378 [219] [DEBUG] [app] Starting request: urn:request:7623497e-849b-410b-9413-053c763d6dca (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:47,378 [222] [DEBUG] [app] Ending request: urn:request:029c6292-fc5f-4371-9ceb-65caa8157e23 (/_internal_ping) gunicorn-web stdout | 2022-05-05 10:21:47,378 [219] [DEBUG] [app] Ending request: urn:request:7623497e-849b-410b-9413-053c763d6dca (/_internal_ping) nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.001 159 0.002) gunicorn-web stdout | 2022-05-05 10:21:47,378 [222] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" gunicorn-web stdout | 2022-05-05 10:21:47,379 [219] [INFO] [gunicorn.access] 127.0.0.1 - - [05/May/2022:10:21:47 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.22.0" nginx stdout | 127.0.0.1 () - - [05/May/2022:10:21:47 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.22.0" (0.002 159 0.002) gunicorn-web stdout | 2022-05-05 10:21:47,379 [221] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:47,379 [222] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4 gunicorn-web stdout | 2022-05-05 10:21:47,379 [222] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:21:47,379 [221] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2022-05-05 10:21:47,386 [221] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:21:47,386 [222] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1]) gunicorn-web stdout | 2022-05-05 10:21:47,392 [221] [DEBUG] [app] Ending request: urn:request:52f6d07f-db9e-4f62-b440-adeee38543be (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:47,392 [221] [DEBUG] [data.database] Disconnecting from database. gunicorn-web stdout | 2022-05-05 10:21:47,393 [221] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:21:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22" nginx stdout | 10.130.2.1 () - - [05/May/2022:10:21:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.024 118 0.024) gunicorn-web stdout | 2022-05-05 10:21:47,393 [222] [DEBUG] [app] Ending request: urn:request:64feabe6-cf70-4e90-8904-86738f9c8e1c (/health/instance) gunicorn-web stdout | 2022-05-05 10:21:47,393 [222] [DEBUG] [data.database] Disconnecting from database. nginx stdout | 10.130.2.1 () - - [05/May/2022:10:21:47 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.22" (0.025 118 0.025) gunicorn-web stdout | 2022-05-05 10:21:47,394 [222] [INFO] [gunicorn.access] 10.130.2.1 - - [05/May/2022:10:21:47 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.22"