__   __
  /  \ /  \     ______   _    _     __   __   __
 / /\ / /\ \   /  __  \ | |  | |   /  \  \ \ / /
/ /  / /  \ \  | |  | | | |  | |  / /\ \  \   /
\ \  \ \  / /  | |__| | | |__| | / ____ \  | |
 \ \/ \ \/ /   \_  ___/  \____/ /_/    \_\ |_|
  \__/ \__/      \ \__
                  \___\ by Red Hat
 Build, Store, and Distribute your Containers

Startup timestamp: 
Fri Jun 14 05:16:35 UTC 2024

Running all default registry services without migration
Running init script '/quay-registry/conf/init/certs_install.sh'
Installing extra certificates found in /quay-registry/conf/stack/extra_ca_certs directory
Running init script '/quay-registry/conf/init/client_certs.sh'
Running init script '/quay-registry/conf/init/copy_config_files.sh'
Running init script '/quay-registry/conf/init/d_validate_config_bundle.sh'
Validating Configuration
time="2024-06-14T05:16:36Z" level=debug msg="Validating AccessSettings"
time="2024-06-14T05:16:36Z" level=debug msg="Validating ActionLogArchiving"
time="2024-06-14T05:16:36Z" level=debug msg="Validating AppTokenAuthentication"
time="2024-06-14T05:16:36Z" level=debug msg="Validating BitbucketBuildTrigger"
time="2024-06-14T05:16:36Z" level=debug msg="Validating BuildManager"
time="2024-06-14T05:16:36Z" level=debug msg="Validating Database"
time="2024-06-14T05:16:36Z" level=debug msg="Scheme: postgresql"
time="2024-06-14T05:16:36Z" level=debug msg="Host: quayregistry-quay-database:5432"
time="2024-06-14T05:16:36Z" level=debug msg="Db: quayregistry-quay-database"
time="2024-06-14T05:16:36Z" level=debug msg="Params: "
time="2024-06-14T05:16:36Z" level=debug msg="Including params "
time="2024-06-14T05:16:36Z" level=debug msg="Pinging database at postgresql://quayregistry-quay-database:JjLKb3j553myXo58rNLqC6oCS9kucp1t-6HqWcHpWuNSzenoFI6p0Nglrp2RHT1VvsYwt1SsW1a4M3eZ@quayregistry-quay-database:5432/quayregistry-quay-database"
plpgsql
pg_trgm
time="2024-06-14T05:16:36Z" level=debug msg="Validating DistributedStorage"
time="2024-06-14T05:16:36Z" level=debug msg="Validating ElasticSearch"
time="2024-06-14T05:16:36Z" level=debug msg="Validating Email"
time="2024-06-14T05:16:36Z" level=debug msg="Validating GitHubBuildTrigger"
time="2024-06-14T05:16:36Z" level=debug msg="Validating GitHubLogin"
time="2024-06-14T05:16:36Z" level=debug msg="Validating GitLabBuildTrigger"
time="2024-06-14T05:16:36Z" level=debug msg="Validating GoogleLogin"
time="2024-06-14T05:16:36Z" level=debug msg="Validating HostSettings"
time="2024-06-14T05:16:36Z" level=debug msg="Validating JWTAuthentication"
time="2024-06-14T05:16:36Z" level=debug msg="Validating LDAP"
time="2024-06-14T05:16:36Z" level=debug msg="Validating OIDC"
time="2024-06-14T05:16:36Z" level=debug msg="Validating QuayDocumentation"
time="2024-06-14T05:16:36Z" level=debug msg="Validating Redis"
time="2024-06-14T05:16:36Z" level=debug msg="Address: quayregistry-quay-redis:6379"
time="2024-06-14T05:16:36Z" level=debug msg="Username: "
time="2024-06-14T05:16:36Z" level=debug msg="Password Len: 0"
time="2024-06-14T05:16:36Z" level=debug msg="Ssl: <nil>"
time="2024-06-14T05:16:36Z" level=debug msg="Address: quayregistry-quay-redis:6379"
time="2024-06-14T05:16:36Z" level=debug msg="Username: "
time="2024-06-14T05:16:36Z" level=debug msg="Password Len: 0"
time="2024-06-14T05:16:36Z" level=debug msg="Ssl: <nil>"
time="2024-06-14T05:16:36Z" level=debug msg="Validating RepoMirror"
time="2024-06-14T05:16:36Z" level=debug msg="Validating SecurityScanner"
time="2024-06-14T05:16:36Z" level=debug msg="Validating TeamSyncing"
time="2024-06-14T05:16:36Z" level=debug msg="Validating TimeMachine"
time="2024-06-14T05:16:36Z" level=debug msg="Validating UserVisibleSettings"
+------------------------+-------+--------+
|      Field Group       | Error | Status |
+------------------------+-------+--------+
| AccessSettings         | -     | 🟢     |
+------------------------+-------+--------+
| ActionLogArchiving     | -     | 🟢     |
+------------------------+-------+--------+
| AppTokenAuthentication | -     | 🟢     |
+------------------------+-------+--------+
| BitbucketBuildTrigger  | -     | 🟢     |
+------------------------+-------+--------+
| BuildManager           | -     | 🟢     |
+------------------------+-------+--------+
| Database               | -     | 🟢     |
+------------------------+-------+--------+
| DistributedStorage     | -     | 🟢     |
+------------------------+-------+--------+
| ElasticSearch          | -     | 🟢     |
+------------------------+-------+--------+
| Email                  | -     | 🟢     |
+------------------------+-------+--------+
| GitHubBuildTrigger     | -     | 🟢     |
+------------------------+-------+--------+
| GitHubLogin            | -     | 🟢     |
+------------------------+-------+--------+
| GitLabBuildTrigger     | -     | 🟢     |
+------------------------+-------+--------+
| GoogleLogin            | -     | 🟢     |
+------------------------+-------+--------+
| HostSettings           | -     | 🟢     |
+------------------------+-------+--------+
| JWTAuthentication      | -     | 🟢     |
+------------------------+-------+--------+
| LDAP                   | -     | 🟢     |
+------------------------+-------+--------+
| OIDC                   | -     | 🟢     |
+------------------------+-------+--------+
| QuayDocumentation      | -     | 🟢     |
+------------------------+-------+--------+
| Redis                  | -     | 🟢     |
+------------------------+-------+--------+
| RepoMirror             | -     | 🟢     |
+------------------------+-------+--------+
| SecurityScanner        | -     | 🟢     |
+------------------------+-------+--------+
| TeamSyncing            | -     | 🟢     |
+------------------------+-------+--------+
| TimeMachine            | -     | 🟢     |
+------------------------+-------+--------+
| UserVisibleSettings    | -     | 🟢     |
+------------------------+-------+--------+
Running init script '/quay-registry/conf/init/nginx_conf_create.sh'
Running init script '/quay-registry/conf/init/supervisord_conf_create.sh'
Running init script '/quay-registry/conf/init/zz_boot.sh'
/app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if word_sep is not '':
2024-06-14 05:16:39,755 INFO RPC interface 'supervisor' initialized
2024-06-14 05:16:39,756 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-06-14 05:16:39,756 INFO supervisord started with pid 7
2024-06-14 05:16:40,759 INFO spawned: 'stdout' with pid 58
2024-06-14 05:16:40,761 INFO spawned: 'autopruneworker' with pid 59
2024-06-14 05:16:40,763 INFO spawned: 'blobuploadcleanupworker' with pid 60
2024-06-14 05:16:40,765 INFO spawned: 'builder' with pid 61
2024-06-14 05:16:40,767 INFO spawned: 'buildlogsarchiver' with pid 62
2024-06-14 05:16:40,768 INFO spawned: 'chunkcleanupworker' with pid 63
2024-06-14 05:16:40,770 INFO spawned: 'dnsmasq' with pid 64
2024-06-14 05:16:40,772 INFO spawned: 'expiredappspecifictokenworker' with pid 65
2024-06-14 05:16:40,775 INFO spawned: 'exportactionlogsworker' with pid 66
2024-06-14 05:16:40,777 INFO spawned: 'gcworker' with pid 67
2024-06-14 05:16:40,779 INFO spawned: 'globalpromstats' with pid 68
2024-06-14 05:16:40,781 INFO spawned: 'gunicorn-registry' with pid 69
2024-06-14 05:16:40,784 INFO spawned: 'gunicorn-secscan' with pid 70
2024-06-14 05:16:40,786 INFO spawned: 'gunicorn-web' with pid 71
2024-06-14 05:16:40,791 INFO spawned: 'logrotateworker' with pid 72
2024-06-14 05:16:40,793 INFO spawned: 'manifestbackfillworker' with pid 73
2024-06-14 05:16:40,796 INFO spawned: 'memcache' with pid 74
2024-06-14 05:16:40,799 INFO spawned: 'namespacegcworker' with pid 75
2024-06-14 05:16:40,888 INFO spawned: 'nginx' with pid 76
2024-06-14 05:16:40,891 INFO spawned: 'notificationworker' with pid 82
2024-06-14 05:16:40,895 INFO spawned: 'pushgateway' with pid 83
2024-06-14 05:16:40,897 INFO spawned: 'queuecleanupworker' with pid 84
2024-06-14 05:16:40,988 INFO spawned: 'quotaregistrysizeworker' with pid 87
2024-06-14 05:16:40,992 INFO spawned: 'quotatotalworker' with pid 88
2024-06-14 05:16:40,996 INFO spawned: 'reconciliationworker' with pid 94
2024-06-14 05:16:40,998 INFO spawned: 'repositoryactioncounter' with pid 108
2024-06-14 05:16:41,001 INFO spawned: 'repositorygcworker' with pid 109
2024-06-14 05:16:41,091 INFO spawned: 'securityscanningnotificationworker' with pid 110
2024-06-14 05:16:41,097 INFO spawned: 'securityworker' with pid 111
2024-06-14 05:16:41,191 INFO spawned: 'servicekey' with pid 112
2024-06-14 05:16:41,195 INFO spawned: 'storagereplication' with pid 113
2024-06-14 05:16:41,198 INFO spawned: 'teamsyncworker' with pid 114
2024-06-14 05:16:42,492 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,492 INFO success: autopruneworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: blobuploadcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: builder entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: buildlogsarchiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: chunkcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: expiredappspecifictokenworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: exportactionlogsworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: gcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: globalpromstats entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: gunicorn-registry entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,493 INFO success: gunicorn-secscan entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: gunicorn-web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: logrotateworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: manifestbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: memcache entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: namespacegcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: notificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: pushgateway entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: queuecleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: quotaregistrysizeworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: quotatotalworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,494 INFO success: reconciliationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,497 INFO success: repositoryactioncounter entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,497 INFO success: repositorygcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,497 INFO success: securityscanningnotificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,497 INFO success: securityworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,497 INFO success: servicekey entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,497 INFO success: storagereplication entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-14 05:16:42,497 INFO success: teamsyncworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
dnsmasq stderr | dnsmasq: started, version 2.79 cachesize 150
dnsmasq stderr | dnsmasq: compile time options: IPv6 GNU-getopt DBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify
dnsmasq stderr | dnsmasq: reading /etc/resolv.conf
dnsmasq stderr | dnsmasq: using nameserver 172.30.0.10#53
dnsmasq stderr | dnsmasq: read /etc/hosts - 7 addresses
nginx stdout | 2024/06/14 05:16:40 [alert] 91#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 106#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 100#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 99#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 102#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 105#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 97#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 96#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 93#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 104#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 107#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 101#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 103#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 95#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 92#0: setpriority(-10) failed (13: Permission denied)
nginx stdout | 2024/06/14 05:16:40 [alert] 98#0: setpriority(-10) failed (13: Permission denied)
pushgateway stderr | ts=2024-06-14T05:16:41.397Z caller=main.go:86 level=info msg="starting pushgateway" version="(version=, branch=, revision=unknown)"
pushgateway stderr | ts=2024-06-14T05:16:41.397Z caller=main.go:87 level=info build_context="(go=go1.19.13, platform=linux/amd64, user=, date=, tags=unknown)"
pushgateway stderr | ts=2024-06-14T05:16:41.490Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9091
pushgateway stderr | ts=2024-06-14T05:16:41.491Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9091
nginx stdout | 2024/06/14 05:16:50 [crit] 91#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.2, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.129.2.44:8080"
nginx stdout | 2024/06/14 05:16:50 [crit] 91#0: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.2, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.129.2.44:8080"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:16:50 +0000] "GET /health/instance HTTP/1.1" 502 157 "-" "kube-probe/1.28" (0.000 118 0.000 : 0.000)
chunkcleanupworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
chunkcleanupworker stderr |   if word_sep is not '':
queuecleanupworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
queuecleanupworker stderr |   if word_sep is not '':
logrotateworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
logrotateworker stderr |   if word_sep is not '':
nginx stdout | 2024/06/14 05:17:05 [crit] 106#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.2, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.129.2.44:8080"
nginx stdout | 2024/06/14 05:17:05 [crit] 106#0: *4 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.2, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.129.2.44:8080"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:17:05 +0000] "GET /health/instance HTTP/1.1" 502 157 "-" "kube-probe/1.28" (0.000 118 0.000 : 0.000)
repositoryactioncounter stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
repositoryactioncounter stderr |   if word_sep is not '':
exportactionlogsworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
exportactionlogsworker stderr |   if word_sep is not '':
expiredappspecifictokenworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
expiredappspecifictokenworker stderr |   if word_sep is not '':
repositorygcworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
repositorygcworker stderr |   if word_sep is not '':
blobuploadcleanupworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
blobuploadcleanupworker stderr |   if word_sep is not '':
servicekey stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
servicekey stderr |   if word_sep is not '':
chunkcleanupworker stdout | 2024-06-14 05:17:10,997 [63] [DEBUG] [__main__] Swift storage not detected; sleeping
securityworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
securityworker stderr |   if word_sep is not '':
queuecleanupworker stdout | 2024-06-14 05:17:11,989 [84] [DEBUG] [workers.worker] Scheduling worker.
queuecleanupworker stdout | 2024-06-14 05:17:11,991 [84] [INFO] [apscheduler.scheduler] Scheduler started
queuecleanupworker stdout | 2024-06-14 05:17:12,001 [84] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
queuecleanupworker stdout | 2024-06-14 05:17:12,001 [84] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
queuecleanupworker stdout | 2024-06-14 05:17:12,089 [84] [DEBUG] [workers.worker] First run scheduled for 2024-06-15 00:37:41.090322
queuecleanupworker stdout | 2024-06-14 05:17:12,090 [84] [INFO] [apscheduler.scheduler] Added job "QueueCleanupWorker._cleanup_queue" to job store "default"
queuecleanupworker stdout | 2024-06-14 05:17:12,099 [84] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
queuecleanupworker stdout | 2024-06-14 05:17:12,100 [84] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-15 00:37:41.090322+00:00 (in 69628.989680 seconds)
manifestbackfillworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
manifestbackfillworker stderr |   if word_sep is not '':
gunicorn-web stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
gunicorn-web stderr |   if word_sep is not '':
gcworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
gcworker stderr |   if word_sep is not '':
namespacegcworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
namespacegcworker stderr |   if word_sep is not '':
logrotateworker stdout | 2024-06-14 05:17:15,893 [72] [DEBUG] [__main__] Action log rotation worker not enabled; skipping
globalpromstats stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
globalpromstats stderr |   if word_sep is not '':
notificationworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
notificationworker stderr |   if word_sep is not '':
quotatotalworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
quotatotalworker stderr |   if word_sep is not '':
quotaregistrysizeworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
quotaregistrysizeworker stderr |   if word_sep is not '':
repositorygcworker stdout | 2024-06-14 05:17:17,990 [109] [DEBUG] [__main__] Starting repository GC worker
repositorygcworker stdout | 2024-06-14 05:17:17,993 [109] [DEBUG] [workers.worker] Scheduling worker.
repositorygcworker stdout | 2024-06-14 05:17:17,994 [109] [INFO] [apscheduler.scheduler] Scheduler started
repositorygcworker stdout | 2024-06-14 05:17:17,995 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:17:17,995 [109] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:34.996948
repositorygcworker stdout | 2024-06-14 05:17:17,996 [109] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
repositorygcworker stdout | 2024-06-14 05:17:17,997 [109] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default"
repositorygcworker stdout | 2024-06-14 05:17:18,088 [109] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:21:05.089356
repositorygcworker stdout | 2024-06-14 05:17:18,088 [109] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default"
repositorygcworker stdout | 2024-06-14 05:17:18,088 [109] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:42.089819
repositorygcworker stdout | 2024-06-14 05:17:18,089 [109] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default"
repositorygcworker stdout | 2024-06-14 05:17:18,089 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:17:18,089 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:34.996948+00:00 (in 16.907544 seconds)
teamsyncworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
teamsyncworker stderr |   if word_sep is not '':
builder stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
builder stderr |   if word_sep is not '':
buildlogsarchiver stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
buildlogsarchiver stderr |   if word_sep is not '':
gunicorn-web stdout | 2024-06-14 05:17:18,890 [71] [DEBUG] [app] Loading default config.
gunicorn-web stdout | 2024-06-14 05:17:18,891 [71] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml
gunicorn-web stdout | 2024-06-14 05:17:18,994 [71] [DEBUG] [app] Loaded config
gunicorn-web stdout | 2024-06-14 05:17:18,995 [71] [INFO] [util.ipresolver] Loading AWS IP ranges from disk
gunicorn-web stdout | 2024-06-14 05:17:19,089 [71] [DEBUG] [util.ipresolver] Building AWS IP ranges
servicekey stdout | 2024-06-14 05:17:19,493 [112] [DEBUG] [workers.worker] Scheduling worker.
servicekey stdout | 2024-06-14 05:17:19,494 [112] [INFO] [apscheduler.scheduler] Scheduler started
servicekey stdout | 2024-06-14 05:17:19,688 [112] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
servicekey stdout | 2024-06-14 05:17:19,688 [112] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:43:55.689609
servicekey stdout | 2024-06-14 05:17:19,689 [112] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
servicekey stdout | 2024-06-14 05:17:19,690 [112] [INFO] [apscheduler.scheduler] Added job "ServiceKeyWorker._refresh_service_key" to job store "default"
servicekey stdout | 2024-06-14 05:17:19,691 [112] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
servicekey stdout | 2024-06-14 05:17:19,691 [112] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:43:55.689609+00:00 (in 1595.998482 seconds)
repositoryactioncounter stdout | 2024-06-14 05:17:19,691 [108] [DEBUG] [workers.worker] Scheduling worker.
repositoryactioncounter stdout | 2024-06-14 05:17:19,692 [108] [INFO] [apscheduler.scheduler] Scheduler started
repositoryactioncounter stdout | 2024-06-14 05:17:19,692 [108] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositoryactioncounter stdout | 2024-06-14 05:17:19,692 [108] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 06:54:03.693857
repositoryactioncounter stdout | 2024-06-14 05:17:19,693 [108] [INFO] [apscheduler.scheduler] Added job "RepositoryActionCountWorker._run_counting" to job store "default"
repositoryactioncounter stdout | 2024-06-14 05:17:19,693 [108] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
repositoryactioncounter stdout | 2024-06-14 05:17:19,694 [108] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositoryactioncounter stdout | 2024-06-14 05:17:19,694 [108] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 06:54:03.693857+00:00 (in 5803.999675 seconds)
gunicorn-web stdout | 2024-06-14 05:17:20,099 [71] [DEBUG] [util.ipresolver] Finished building AWS IP ranges
nginx stdout | 2024/06/14 05:17:20 [crit] 100#0: *7 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.2, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.129.2.44:8080"
nginx stdout | 2024/06/14 05:17:20 [crit] 100#0: *7 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.129.2.2, server: , request: "GET /health/instance HTTP/1.1", upstream: "http://unix:/tmp/gunicorn_web.sock:/quay-registry/static/502.html", host: "10.129.2.44:8080"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:17:20 +0000] "GET /health/instance HTTP/1.1" 502 157 "-" "kube-probe/1.28" (0.000 118 0.000 : 0.000)
blobuploadcleanupworker stdout | 2024-06-14 05:17:20,492 [60] [DEBUG] [workers.worker] Scheduling worker.
blobuploadcleanupworker stdout | 2024-06-14 05:17:20,492 [60] [INFO] [apscheduler.scheduler] Scheduler started
reconciliationworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
reconciliationworker stderr |   if word_sep is not '':
gunicorn-web stdout | 2024-06-14 05:17:20,495 [71] [DEBUG] [data.database] Configuring database
gunicorn-web stdout | 2024-06-14 05:17:20,497 [71] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:17:20,499 [71] [INFO] [data.secscan_model] ===============================
blobuploadcleanupworker stdout | 2024-06-14 05:17:20,588 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
blobuploadcleanupworker stdout | 2024-06-14 05:17:20,588 [60] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
gunicorn-web stdout | 2024-06-14 05:17:20,590 [71] [INFO] [data.secscan_model] Using split secscan model: `[<data.secscan_model.secscan_v4_model.V4SecurityScanner object at 0x7fcdabe1bf10>]`
gunicorn-web stdout | 2024-06-14 05:17:20,591 [71] [INFO] [data.secscan_model] ===============================
gunicorn-web stdout | 2024-06-14 05:17:20,592 [71] [DEBUG] [data.logs_model] Configuring log model
gunicorn-web stdout | 2024-06-14 05:17:20,592 [71] [INFO] [data.logs_model] ===============================
gunicorn-web stdout | 2024-06-14 05:17:20,593 [71] [INFO] [data.logs_model] Using logs model `<data.logs_model.table_logs_model.TableLogsModel object at 0x7fcdab449760>`
gunicorn-web stdout | 2024-06-14 05:17:20,594 [71] [INFO] [data.logs_model] ===============================
blobuploadcleanupworker stdout | 2024-06-14 05:17:20,597 [60] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 06:06:27.598395
blobuploadcleanupworker stdout | 2024-06-14 05:17:20,598 [60] [INFO] [apscheduler.scheduler] Added job "BlobUploadCleanupWorker._try_cleanup_uploads" to job store "default"
blobuploadcleanupworker stdout | 2024-06-14 05:17:20,598 [60] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
blobuploadcleanupworker stdout | 2024-06-14 05:17:20,598 [60] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 06:06:27.598395+00:00 (in 2946.999527 seconds)
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,796 [65] [DEBUG] [__main__] Starting expired app specific token GC worker
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,796 [65] [DEBUG] [__main__] Found expiration window: 1d
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,798 [65] [DEBUG] [workers.worker] Scheduling worker.
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,798 [65] [INFO] [apscheduler.scheduler] Scheduler started
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,901 [65] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,901 [65] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,988 [65] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:28:16.989264
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,989 [65] [INFO] [apscheduler.scheduler] Added job "ExpiredAppSpecificTokenWorker._gc_expired_tokens" to job store "default"
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,989 [65] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
expiredappspecifictokenworker stdout | 2024-06-14 05:17:20,990 [65] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:28:16.989264+00:00 (in 655.999101 seconds)
storagereplication stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
storagereplication stderr |   if word_sep is not '':
autopruneworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
autopruneworker stderr |   if word_sep is not '':
manifestbackfillworker stdout | 2024-06-14 05:17:24,595 [73] [DEBUG] [workers.worker] Scheduling worker.
manifestbackfillworker stdout | 2024-06-14 05:17:24,597 [73] [INFO] [apscheduler.scheduler] Scheduler started
namespacegcworker stdout | 2024-06-14 05:17:24,696 [75] [DEBUG] [__main__] Starting namespace GC worker
namespacegcworker stdout | 2024-06-14 05:17:24,700 [75] [DEBUG] [workers.worker] Scheduling worker.
namespacegcworker stdout | 2024-06-14 05:17:24,701 [75] [INFO] [apscheduler.scheduler] Scheduler started
manifestbackfillworker stdout | 2024-06-14 05:17:24,788 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
manifestbackfillworker stdout | 2024-06-14 05:17:24,788 [73] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
manifestbackfillworker stdout | 2024-06-14 05:17:24,789 [73] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 06:13:06.790493
manifestbackfillworker stdout | 2024-06-14 05:17:24,790 [73] [INFO] [apscheduler.scheduler] Added job "ManifestBackfillWorker._backfill_manifests" to job store "default"
manifestbackfillworker stdout | 2024-06-14 05:17:24,791 [73] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
manifestbackfillworker stdout | 2024-06-14 05:17:24,791 [73] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 06:13:06.790493+00:00 (in 3341.999149 seconds)
gcworker stdout | 2024-06-14 05:17:24,793 [67] [DEBUG] [workers.worker] Scheduling worker.
gcworker stdout | 2024-06-14 05:17:24,795 [67] [INFO] [apscheduler.scheduler] Scheduler started
namespacegcworker stdout | 2024-06-14 05:17:24,797 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:17:24,797 [75] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:18:03.798679
namespacegcworker stdout | 2024-06-14 05:17:24,799 [75] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default"
namespacegcworker stdout | 2024-06-14 05:17:24,799 [75] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:20:08.800520
namespacegcworker stdout | 2024-06-14 05:17:24,799 [75] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default"
namespacegcworker stdout | 2024-06-14 05:17:24,799 [75] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:18:15.800790
namespacegcworker stdout | 2024-06-14 05:17:24,799 [75] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default"
namespacegcworker stdout | 2024-06-14 05:17:24,798 [75] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
namespacegcworker stdout | 2024-06-14 05:17:24,800 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:17:24,800 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:03.798679+00:00 (in 38.998016 seconds)
gcworker stdout | 2024-06-14 05:17:24,988 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:17:24,988 [67] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
gcworker stdout | 2024-06-14 05:17:24,990 [67] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:32.991899
gcworker stdout | 2024-06-14 05:17:24,992 [67] [INFO] [apscheduler.scheduler] Added job "GarbageCollectionWorker._garbage_collection_repos" to job store "default"
gcworker stdout | 2024-06-14 05:17:24,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:17:24,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:32.991899+00:00 (in 7.999090 seconds)
securityscanningnotificationworker stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
securityscanningnotificationworker stderr |   if word_sep is not '':
securityworker stdout | 2024-06-14 05:17:26,997 [111] [DEBUG] [workers.worker] Scheduling worker.
securityworker stdout | 2024-06-14 05:17:26,999 [111] [INFO] [apscheduler.scheduler] Scheduler started
teamsyncworker stdout | 2024-06-14 05:17:27,095 [114] [DEBUG] [__main__] Team syncing is disabled; sleeping
securityworker stdout | 2024-06-14 05:17:27,188 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:17:27,188 [111] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
securityworker stdout | 2024-06-14 05:17:27,190 [111] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:46.191483
securityworker stdout | 2024-06-14 05:17:27,192 [111] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_in_scanner" to job store "default"
securityworker stdout | 2024-06-14 05:17:27,193 [111] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:28.194620
securityworker stdout | 2024-06-14 05:17:27,193 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:17:27,195 [87] [DEBUG] [workers.worker] Scheduling worker.
securityworker stdout | 2024-06-14 05:17:27,194 [111] [INFO] [apscheduler.scheduler] Added job "SecurityWorker._index_recent_manifests_in_scanner" to job store "default"
securityworker stdout | 2024-06-14 05:17:27,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:28.194620+00:00 (in 0.998710 seconds)
securityworker stdout | 2024-06-14 05:17:27,196 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:17:27,196 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:28.194620+00:00 (in 0.998528 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:17:27,197 [87] [INFO] [apscheduler.scheduler] Scheduler started
quotaregistrysizeworker stdout | 2024-06-14 05:17:27,197 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:17:27,197 [87] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:18:07.198655
quotaregistrysizeworker stdout | 2024-06-14 05:17:27,198 [87] [INFO] [apscheduler.scheduler] Added job "QuotaRegistrySizeWorker._calculate_registry_size" to job store "default"
quotaregistrysizeworker stdout | 2024-06-14 05:17:27,198 [87] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
quotaregistrysizeworker stdout | 2024-06-14 05:17:27,199 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:17:27,199 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:07.198655+00:00 (in 39.999576 seconds)
gunicorn-secscan stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
gunicorn-secscan stderr |   if word_sep is not '':
globalpromstats stdout | 2024-06-14 05:17:27,492 [68] [DEBUG] [workers.worker] Scheduling worker.
globalpromstats stdout | 2024-06-14 05:17:27,493 [68] [INFO] [apscheduler.scheduler] Scheduler started
globalpromstats stdout | 2024-06-14 05:17:27,493 [68] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
globalpromstats stdout | 2024-06-14 05:17:27,493 [68] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
globalpromstats stdout | 2024-06-14 05:17:27,494 [68] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:23:04.495040
globalpromstats stdout | 2024-06-14 05:17:27,495 [68] [INFO] [apscheduler.scheduler] Added job "GlobalPrometheusStatsWorker._try_report_stats" to job store "default"
globalpromstats stdout | 2024-06-14 05:17:27,495 [68] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
globalpromstats stdout | 2024-06-14 05:17:27,495 [68] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:04.495040+00:00 (in 336.999792 seconds)
notificationworker stdout | 2024-06-14 05:17:27,503 [82] [DEBUG] [workers.worker] Scheduling worker.
notificationworker stdout | 2024-06-14 05:17:27,504 [82] [INFO] [apscheduler.scheduler] Scheduler started
notificationworker stdout | 2024-06-14 05:17:27,589 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:17:27,589 [82] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
notificationworker stdout | 2024-06-14 05:17:27,590 [82] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:31.591084
notificationworker stdout | 2024-06-14 05:17:27,591 [82] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default"
notificationworker stdout | 2024-06-14 05:17:27,591 [82] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:19:36.592214
notificationworker stdout | 2024-06-14 05:17:27,591 [82] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default"
notificationworker stdout | 2024-06-14 05:17:27,591 [82] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:18:21.592513
notificationworker stdout | 2024-06-14 05:17:27,591 [82] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default"
notificationworker stdout | 2024-06-14 05:17:27,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:17:27,592 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:31.591084+00:00 (in 3.999078 seconds)
buildlogsarchiver stdout | 2024-06-14 05:17:27,889 [62] [DEBUG] [workers.worker] Scheduling worker.
buildlogsarchiver stdout | 2024-06-14 05:17:27,889 [62] [INFO] [apscheduler.scheduler] Scheduler started
buildlogsarchiver stdout | 2024-06-14 05:17:27,890 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:17:27,890 [62] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
buildlogsarchiver stdout | 2024-06-14 05:17:27,890 [62] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:31.891735
buildlogsarchiver stdout | 2024-06-14 05:17:27,891 [62] [INFO] [apscheduler.scheduler] Added job "ArchiveBuildLogsWorker._archive_redis_buildlogs" to job store "default"
buildlogsarchiver stdout | 2024-06-14 05:17:27,891 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:17:27,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:31.891735+00:00 (in 3.999735 seconds)
exportactionlogsworker stdout | 2024-06-14 05:17:28,192 [66] [DEBUG] [__main__] Starting export action logs worker
securityworker stdout | 2024-06-14 05:17:28,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:17:28,195 [66] [DEBUG] [workers.worker] Scheduling worker.
exportactionlogsworker stdout | 2024-06-14 05:17:28,195 [66] [INFO] [apscheduler.scheduler] Scheduler started
securityworker stdout | 2024-06-14 05:17:28,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:17:28 UTC)" (scheduled at 2024-06-14 05:17:28.194620+00:00)
exportactionlogsworker stdout | 2024-06-14 05:17:28,196 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:17:28,196 [66] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
securityworker stdout | 2024-06-14 05:17:28,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:17:28,196 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
exportactionlogsworker stdout | 2024-06-14 05:17:28,196 [66] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:18:23.197203
securityworker stdout | 2024-06-14 05:17:28,196 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:46.191483+00:00 (in 17.995202 seconds)
securityworker stdout | 2024-06-14 05:17:28,196 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:17:28,196 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
exportactionlogsworker stdout | 2024-06-14 05:17:28,197 [66] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default"
exportactionlogsworker stdout | 2024-06-14 05:17:28,197 [66] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:21:08.198104
exportactionlogsworker stdout | 2024-06-14 05:17:28,197 [66] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default"
exportactionlogsworker stdout | 2024-06-14 05:17:28,197 [66] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:18:06.198336
securityworker stdout | 2024-06-14 05:17:28,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:17:28,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
exportactionlogsworker stdout | 2024-06-14 05:17:28,197 [66] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default"
exportactionlogsworker stdout | 2024-06-14 05:17:28,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:17:28,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:06.198336+00:00 (in 38.000642 seconds)
securityworker stdout | 2024-06-14 05:17:28,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:17:28,199 [111] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): quayregistry-clair-app.quay-enterprise.svc.cluster.local:80
securityworker stdout | 2024-06-14 05:17:28,215 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:17:28,290 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:17:28,298 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:17:28,300 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:17:28,300 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:17:28,301 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 12, 28, 289295), 1, 2])
securityworker stdout | 2024-06-14 05:17:28,302 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:17:28,302 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:17:28,303 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 12, 28, 289295), 1, 2])
securityworker stdout | 2024-06-14 05:17:28,304 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:17:28,304 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:17:28,304 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:17:28,304 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:17:28,304 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:17:28,305 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:17:28,305 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:17:28,305 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:17:28,305 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:17:28,305 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:17:58 UTC)" executed successfully
autopruneworker stdout | 2024-06-14 05:17:28,801 [59] [DEBUG] [workers.worker] Scheduling worker.
autopruneworker stdout | 2024-06-14 05:17:28,802 [59] [INFO] [apscheduler.scheduler] Scheduler started
autopruneworker stdout | 2024-06-14 05:17:28,802 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:17:28,803 [59] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
autopruneworker stdout | 2024-06-14 05:17:28,803 [59] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:55.804090
autopruneworker stdout | 2024-06-14 05:17:28,804 [59] [INFO] [apscheduler.scheduler] Added job "AutoPruneWorker.prune" to job store "default"
autopruneworker stdout | 2024-06-14 05:17:28,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:17:28,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:55.804090+00:00 (in 26.999662 seconds)
storagereplication stdout | 2024-06-14 05:17:28,906 [113] [DEBUG] [__main__] Full storage replication disabled; skipping
builder stdout | 2024-06-14 05:17:28,909 [61] [DEBUG] [__main__] Building is disabled. Please enable the feature flag
gunicorn-web stdout | 2024-06-14 05:17:29,125 [71] [DEBUG] [__config__] Starting web gunicorn with 4 workers and gevent worker class
gunicorn-secscan stdout | 2024-06-14 05:17:29,315 [70] [DEBUG] [app] Loading default config.
gunicorn-secscan stdout | 2024-06-14 05:17:29,315 [70] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml
gunicorn-secscan stdout | 2024-06-14 05:17:29,323 [70] [DEBUG] [app] Loaded config
gunicorn-secscan stdout | 2024-06-14 05:17:29,323 [70] [INFO] [util.ipresolver] Loading AWS IP ranges from disk
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,327 [110] [DEBUG] [__main__] Starting security scanning notification worker
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,329 [110] [DEBUG] [workers.worker] Scheduling worker.
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,329 [110] [INFO] [apscheduler.scheduler] Scheduler started
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,329 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,330 [110] [DEBUG] [apscheduler.scheduler] No jobs; waiting until a job is added
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,330 [110] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:31.331172
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,330 [110] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default"
gunicorn-secscan stdout | 2024-06-14 05:17:29,331 [70] [DEBUG] [util.ipresolver] Building AWS IP ranges
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,331 [110] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:30.331990
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,331 [110] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default"
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,331 [110] [DEBUG] [workers.worker] First run scheduled for 2024-06-14 05:17:35.332183
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,331 [110] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default"
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:17:29,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:30.331990+00:00 (in 1.000429 seconds)
gunicorn-secscan stdout | 2024-06-14 05:17:29,440 [70] [DEBUG] [util.ipresolver] Finished building AWS IP ranges
gunicorn-secscan stdout | 2024-06-14 05:17:29,526 [70] [DEBUG] [data.database] Configuring database
gunicorn-secscan stdout | 2024-06-14 05:17:29,527 [70] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-secscan stdout | 2024-06-14 05:17:29,527 [70] [INFO] [data.secscan_model] ===============================
gunicorn-secscan stdout | 2024-06-14 05:17:29,527 [70] [INFO] [data.secscan_model] Using split secscan model: `[<data.secscan_model.secscan_v4_model.V4SecurityScanner object at 0x7f2c36dd5e50>]`
gunicorn-secscan stdout | 2024-06-14 05:17:29,528 [70] [INFO] [data.secscan_model] ===============================
gunicorn-secscan stdout | 2024-06-14 05:17:29,528 [70] [DEBUG] [data.logs_model] Configuring log model
gunicorn-secscan stdout | 2024-06-14 05:17:29,528 [70] [INFO] [data.logs_model] ===============================
gunicorn-secscan stdout | 2024-06-14 05:17:29,528 [70] [INFO] [data.logs_model] Using logs model `<data.logs_model.table_logs_model.TableLogsModel object at 0x7f2c36405160>`
gunicorn-secscan stdout | 2024-06-14 05:17:29,528 [70] [INFO] [data.logs_model] ===============================
gunicorn-secscan stdout | 2024-06-14 05:17:29,700 [70] [DEBUG] [__config__] Starting secscan gunicorn with 2 workers and gevent worker class
gunicorn-registry stderr | /app/lib/python3.9/site-packages/netaddr/strategy/__init__.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="?
gunicorn-registry stderr |   if word_sep is not '':
gunicorn-registry stdout | 2024-06-14 05:17:30,178 [69] [DEBUG] [app] Loading default config.
gunicorn-registry stdout | 2024-06-14 05:17:30,179 [69] [DEBUG] [util.config.provider.basefileprovider] Applying config file: /quay-registry/conf/stack/config.yaml
gunicorn-registry stdout | 2024-06-14 05:17:30,186 [69] [DEBUG] [app] Loaded config
gunicorn-registry stdout | 2024-06-14 05:17:30,187 [69] [INFO] [util.ipresolver] Loading AWS IP ranges from disk
gunicorn-registry stdout | 2024-06-14 05:17:30,194 [69] [DEBUG] [util.ipresolver] Building AWS IP ranges
gunicorn-registry stdout | 2024-06-14 05:17:30,271 [69] [DEBUG] [util.ipresolver] Finished building AWS IP ranges
gunicorn-registry stdout | 2024-06-14 05:17:30,310 [69] [DEBUG] [data.database] Configuring database
gunicorn-registry stdout | 2024-06-14 05:17:30,311 [69] [INFO] [data.database] Connection pooling enabled for postgresql; stale timeout: None; max connection count: None
gunicorn-registry stdout | 2024-06-14 05:17:30,312 [69] [INFO] [data.secscan_model] ===============================
gunicorn-registry stdout | 2024-06-14 05:17:30,312 [69] [INFO] [data.secscan_model] Using split secscan model: `[<data.secscan_model.secscan_v4_model.V4SecurityScanner object at 0x7f3b8d46e2b0>]`
gunicorn-registry stdout | 2024-06-14 05:17:30,312 [69] [INFO] [data.secscan_model] ===============================
gunicorn-registry stdout | 2024-06-14 05:17:30,312 [69] [DEBUG] [data.logs_model] Configuring log model
gunicorn-registry stdout | 2024-06-14 05:17:30,312 [69] [INFO] [data.logs_model] ===============================
gunicorn-registry stdout | 2024-06-14 05:17:30,312 [69] [INFO] [data.logs_model] Using logs model `<data.logs_model.table_logs_model.TableLogsModel object at 0x7f3b8d46e4c0>`
gunicorn-registry stdout | 2024-06-14 05:17:30,312 [69] [INFO] [data.logs_model] ===============================
securityscanningnotificationworker stdout | 2024-06-14 05:17:30,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:17:30,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:17:30 UTC)" (scheduled at 2024-06-14 05:17:30.331990+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:17:30,334 [110] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2024, 6, 14, 5, 17, 30, 333132), 'secscanv4/%'])
securityscanningnotificationworker stdout | 2024-06-14 05:17:30,334 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:31.331172+00:00 (in 0.996575 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:17:30,340 [110] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 17, 30, 333132), True, datetime.datetime(2024, 6, 14, 5, 17, 30, 333132), 0, 'secscanv4/%'])
securityscanningnotificationworker stdout | 2024-06-14 05:17:30,341 [110] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 17, 30, 333132), True, datetime.datetime(2024, 6, 14, 5, 17, 30, 333132), 0, 'secscanv4/%', False, datetime.datetime(2024, 6, 14, 5, 17, 30, 333132), 'secscanv4/%'])
securityscanningnotificationworker stdout | 2024-06-14 05:17:30,342 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:17:30,342 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:22:30 UTC)" executed successfully
gunicorn-registry stdout | 2024-06-14 05:17:30,778 [69] [DEBUG] [__config__] Starting registry gunicorn with 8 workers and gevent worker class
securityscanningnotificationworker stdout | 2024-06-14 05:17:31,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:17:31,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:35.332183+00:00 (in 4.000336 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:17:31,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:31 UTC)" (scheduled at 2024-06-14 05:17:31.331172+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:17:31,332 [110] [DEBUG] [workers.queueworker] Getting work item from queue.
securityscanningnotificationworker stdout | 2024-06-14 05:17:31,333 [110] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 17, 31, 332303), True, datetime.datetime(2024, 6, 14, 5, 17, 31, 332303), 0, 'secscanv4/%', 50, 1, 0])
securityscanningnotificationworker stdout | 2024-06-14 05:17:31,339 [110] [DEBUG] [workers.queueworker] No more work.
securityscanningnotificationworker stdout | 2024-06-14 05:17:31,339 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:17:31,339 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:31 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:17:31,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:17:31,592 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:17:31 UTC)" (scheduled at 2024-06-14 05:17:31.591084+00:00)
notificationworker stdout | 2024-06-14 05:17:31,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:17:31,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 17, 31, 592289), True, datetime.datetime(2024, 6, 14, 5, 17, 31, 592289), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:17:31,593 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:41.591084+00:00 (in 9.997274 seconds)
notificationworker stdout | 2024-06-14 05:17:31,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:17:31,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:17:31,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:17:41 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:17:31,891 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:17:31,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:17:31 UTC)" (scheduled at 2024-06-14 05:17:31.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:17:31,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 17, 31, 892886), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:17:31,894 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:01.891735+00:00 (in 29.997531 seconds)
buildlogsarchiver stdout | 2024-06-14 05:17:31,900 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:17:31,900 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:17:31,900 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:01 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:17:32,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:17:32,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:17:32 UTC)" (scheduled at 2024-06-14 05:17:32.991899+00:00)
gcworker stdout | 2024-06-14 05:17:32,993 [67] [DEBUG] [peewee] ('SELECT DISTINCT "t1"."removed_tag_expiration_s" FROM "user" AS "t1" LIMIT %s', [100])
gcworker stdout | 2024-06-14 05:17:32,993 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:02.991899+00:00 (in 29.998144 seconds)
gcworker stdout | 2024-06-14 05:17:32,999 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132652998, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:17:33,001 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:17:33,001 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:17:33,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:02 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:17:34,997 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:17:34,997 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:17:34 UTC)" (scheduled at 2024-06-14 05:17:34.996948+00:00)
repositorygcworker stdout | 2024-06-14 05:17:34,997 [109] [DEBUG] [workers.queueworker] Getting work item from queue.
repositorygcworker stdout | 2024-06-14 05:17:34,999 [109] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 17, 34, 998023), True, datetime.datetime(2024, 6, 14, 5, 17, 34, 998023), 0, 'repositorygc/%', 50, 1, 0])
repositorygcworker stdout | 2024-06-14 05:17:34,999 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:42.089819+00:00 (in 7.090171 seconds)
repositorygcworker stdout | 2024-06-14 05:17:35,005 [109] [DEBUG] [workers.queueworker] No more work.
repositorygcworker stdout | 2024-06-14 05:17:35,005 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:17:35,005 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:34 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:17:35,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:17:35,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:31.331172+00:00 (in 55.998342 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:17:35,333 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:35 UTC)" (scheduled at 2024-06-14 05:17:35.332183+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:17:35,333 [110] [DEBUG] [workers.queueworker] Running watchdog.
securityscanningnotificationworker stdout | 2024-06-14 05:17:35,333 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:35 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:17:35,382 [271] [DEBUG] [app] Starting request: urn:request:3c52d6a9-0c13-4344-9955-b33537b639a4 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:17:35,387 [271] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080
gunicorn-registry stdout | 2024-06-14 05:17:35,395 [287] [DEBUG] [app] Starting request: urn:request:4b2f288a-6ad4-4939-a3ef-338f8d479e59 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:17:35,396 [287] [DEBUG] [app] Ending request: urn:request:4b2f288a-6ad4-4939-a3ef-338f8d479e59 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4b2f288a-6ad4-4939-a3ef-338f8d479e59', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:17:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.005 162 0.005)
gunicorn-registry stdout | 2024-06-14 05:17:35,397 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:17:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:17:35,398 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:17:35,399 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:17:35,401 [271] [DEBUG] [app] Starting request: urn:request:8588e84d-e5ea-4cbc-8495-994c7992167c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:17:35,401 [271] [DEBUG] [app] Ending request: urn:request:8588e84d-e5ea-4cbc-8495-994c7992167c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8588e84d-e5ea-4cbc-8495-994c7992167c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:17:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:17:35,402 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:17:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:17:35,402 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:17:35,405 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2024, 6, 14, 5, 17, 35, 404105), None, 'quay', 'quay', datetime.datetime(2024, 6, 14, 5, 17, 35, 404136), 'quay', None, datetime.datetime(2024, 6, 13, 5, 17, 35, 404148), datetime.datetime(2024, 6, 7, 5, 17, 35, 404161), None])
gunicorn-web stdout | 2024-06-14 05:17:35,414 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:17:35,418 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:17:35,419 [271] [DEBUG] [app] Ending request: urn:request:3c52d6a9-0c13-4344-9955-b33537b639a4 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3c52d6a9-0c13-4344-9955-b33537b639a4', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:17:35,420 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:17:35,420 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:17:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:17:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.042 118 0.042)
gunicorn-web stdout | 2024-06-14 05:17:35,421 [271] [DEBUG] [app] Starting request: urn:request:85907afc-186f-4840-99ed-015faee75976 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-registry stdout | 2024-06-14 05:17:35,424 [287] [DEBUG] [app] Starting request: urn:request:52736473-e6ba-4f60-965b-10f4f52e128d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:17:35,424 [287] [DEBUG] [app] Ending request: urn:request:52736473-e6ba-4f60-965b-10f4f52e128d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:52736473-e6ba-4f60-965b-10f4f52e128d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:17:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-registry stdout | 2024-06-14 05:17:35,424 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:17:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:17:35,424 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:17:35,426 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:17:35,430 [268] [DEBUG] [app] Starting request: urn:request:3f040499-529b-4ea7-ae95-79d0c9c0682d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:17:35,431 [268] [DEBUG] [app] Ending request: urn:request:3f040499-529b-4ea7-ae95-79d0c9c0682d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3f040499-529b-4ea7-ae95-79d0c9c0682d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:17:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.005 159 0.005)
gunicorn-web stdout | 2024-06-14 05:17:35,431 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:17:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:17:35,432 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:17:35,432 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:17:35,436 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:17:35,440 [271] [DEBUG] [app] Ending request: urn:request:85907afc-186f-4840-99ed-015faee75976 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:85907afc-186f-4840-99ed-015faee75976', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:17:35,440 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:17:35,440 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:17:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:17:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
chunkcleanupworker stdout | 2024-06-14 05:17:38,434 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:17:39,228 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:17:41,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:17:41,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:51.591084+00:00 (in 9.999308 seconds)
notificationworker stdout | 2024-06-14 05:17:41,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:17:51 UTC)" (scheduled at 2024-06-14 05:17:41.591084+00:00)
notificationworker stdout | 2024-06-14 05:17:41,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:17:41,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 17, 41, 592226), True, datetime.datetime(2024, 6, 14, 5, 17, 41, 592226), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:17:41,600 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:17:41,600 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:17:41,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:17:51 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:17:42,090 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:17:42,090 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:34.996948+00:00 (in 52.906619 seconds)
repositorygcworker stdout | 2024-06-14 05:17:42,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:42 UTC)" (scheduled at 2024-06-14 05:17:42.089819+00:00)
repositorygcworker stdout | 2024-06-14 05:17:42,090 [109] [DEBUG] [workers.queueworker] Running watchdog.
repositorygcworker stdout | 2024-06-14 05:17:42,090 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:42 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:17:43,936 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:17:45,224 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
repositorygcworker stdout | 2024-06-14 05:17:45,793 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:17:46,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:17:46,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:17:58.194620+00:00 (in 12.002379 seconds)
securityworker stdout | 2024-06-14 05:17:46,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:16 UTC)" (scheduled at 2024-06-14 05:17:46.191483+00:00)
securityworker stdout | 2024-06-14 05:17:46,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:17:46,193 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:17:46,196 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:17:46,198 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:17:46,203 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:17:46,204 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:17:46,204 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:16 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:17:46,435 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:17:46,930 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:17:48,311 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:17:48,534 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:17:49,003 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:17:49,006 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:17:49,009 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:17:49,010 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:17:49,011 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:17:50,379 [271] [DEBUG] [app] Starting request: urn:request:917dcd3b-6d3b-47c0-9493-17e36b64893d (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:17:50,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:17:50,381 [270] [DEBUG] [app] Starting request: urn:request:9baaf4a6-430a-4e0b-9ef6-695280c8916f (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-registry stdout | 2024-06-14 05:17:50,382 [287] [DEBUG] [app] Starting request: urn:request:1c319817-5a37-4481-9f14-9c175ab049c2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:17:50,383 [287] [DEBUG] [app] Ending request: urn:request:1c319817-5a37-4481-9f14-9c175ab049c2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1c319817-5a37-4481-9f14-9c175ab049c2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:17:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:17:50,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:17:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:17:50,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:17:50,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:17:50,385 [270] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080
gunicorn-web stdout | 2024-06-14 05:17:50,386 [271] [DEBUG] [app] Starting request: urn:request:5c6ce581-ebc6-4e50-9b89-33638557b896 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:17:50,386 [271] [DEBUG] [app] Ending request: urn:request:5c6ce581-ebc6-4e50-9b89-33638557b896 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5c6ce581-ebc6-4e50-9b89-33638557b896', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:17:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:17:50,387 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:17:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:17:50,387 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:17:50,387 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-registry stdout | 2024-06-14 05:17:50,389 [280] [DEBUG] [app] Starting request: urn:request:af3faebf-51eb-47b8-be25-05e5d861b8fc (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:17:50,391 [280] [DEBUG] [app] Ending request: urn:request:af3faebf-51eb-47b8-be25-05e5d861b8fc (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:af3faebf-51eb-47b8-be25-05e5d861b8fc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:17:50,391 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-registry stdout | 2024-06-14 05:17:50,392 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:17:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:17:50,392 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:17:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.006 162 0.006)
gunicorn-web stdout | 2024-06-14 05:17:50,393 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:17:50,395 [270] [DEBUG] [app] Starting request: urn:request:11ba95a3-c627-46c5-99bb-a82a16fccdbe (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:17:50,395 [270] [DEBUG] [app] Ending request: urn:request:11ba95a3-c627-46c5-99bb-a82a16fccdbe (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:11ba95a3-c627-46c5-99bb-a82a16fccdbe', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:17:50,396 [271] [DEBUG] [app] Ending request: urn:request:917dcd3b-6d3b-47c0-9493-17e36b64893d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:917dcd3b-6d3b-47c0-9493-17e36b64893d', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:17:50,396 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:17:50,396 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:17:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:17:50,396 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:17:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:17:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:17:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
gunicorn-web stdout | 2024-06-14 05:17:50,397 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:17:50,398 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2024, 6, 14, 5, 17, 50, 397906), None, 'quay', 'quay', datetime.datetime(2024, 6, 14, 5, 17, 50, 397934), 'quay', None, datetime.datetime(2024, 6, 13, 5, 17, 50, 397944), datetime.datetime(2024, 6, 7, 5, 17, 50, 397952), None])
gunicorn-web stdout | 2024-06-14 05:17:50,406 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:17:50,410 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:17:50,411 [270] [DEBUG] [app] Ending request: urn:request:9baaf4a6-430a-4e0b-9ef6-695280c8916f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9baaf4a6-430a-4e0b-9ef6-695280c8916f', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:17:50,411 [270] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:17:50,411 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:17:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:17:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.034 118 0.034)
notificationworker stdout | 2024-06-14 05:17:51,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:17:51,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:01.591084+00:00 (in 9.999442 seconds)
notificationworker stdout | 2024-06-14 05:17:51,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:01 UTC)" (scheduled at 2024-06-14 05:17:51.591084+00:00)
notificationworker stdout | 2024-06-14 05:17:51,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:17:51,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 17, 51, 592041), True, datetime.datetime(2024, 6, 14, 5, 17, 51, 592041), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:17:51,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:17:51,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:17:51,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:01 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:17:51,823 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:17:52,223 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:17:52,623 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:17:52,708 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
quotaregistrysizeworker stdout | 2024-06-14 05:17:54,831 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:17:55,111 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:17:55,504 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:17:55,702 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
autopruneworker stdout | 2024-06-14 05:17:55,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:17:55,805 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:17:55 UTC)" (scheduled at 2024-06-14 05:17:55.804090+00:00)
autopruneworker stdout | 2024-06-14 05:17:55,805 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:25.804090+00:00 (in 29.998397 seconds)
autopruneworker stdout | 2024-06-14 05:17:55,810 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342215809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:17:55,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:17:55,813 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:17:55,813 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:25 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:17:56,903 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:17:57,529 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:17:57,608 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:17:57,936 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:17:58,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:17:58,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:16.191483+00:00 (in 17.996215 seconds)
securityworker stdout | 2024-06-14 05:17:58,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:28 UTC)" (scheduled at 2024-06-14 05:17:58.194620+00:00)
securityworker stdout | 2024-06-14 05:17:58,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:17:58,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:17:58,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:17:58,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:17:58,198 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:17:58,198 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:17:58,198 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:17:58,203 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:17:58,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:17:58,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:17:58,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:17:58,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:17:58,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 12, 58, 204385), 1, 2])
securityworker stdout | 2024-06-14 05:17:58,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:17:58,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:17:58,213 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 12, 58, 204385), 1, 2])
securityworker stdout | 2024-06-14 05:17:58,213 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:17:58,213 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:17:58,213 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:17:58,213 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:17:58,213 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:17:58,214 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:17:58,214 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:17:58,214 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:17:58,214 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:17:58,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:28 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:17:58,925 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:17:59,338 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:17:59,340 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:17:59,342 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:18:00,201 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:18:00,204 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:18:00,206 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:18:00,209 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:18:00,213 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:18:00,215 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:18:00,219 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:18:00,221 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
gunicorn-registry stdout | 2024-06-14 05:18:00,224 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
notificationworker stdout | 2024-06-14 05:18:01,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:18:01,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:11.591084+00:00 (in 9.999417 seconds)
notificationworker stdout | 2024-06-14 05:18:01,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:11 UTC)" (scheduled at 2024-06-14 05:18:01.591084+00:00)
notificationworker stdout | 2024-06-14 05:18:01,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:18:01,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 1, 592097), True, datetime.datetime(2024, 6, 14, 5, 18, 1, 592097), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:18:01,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:18:01,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:18:01,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:11 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:18:01,891 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:18:01,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:31.891735+00:00 (in 29.999471 seconds)
buildlogsarchiver stdout | 2024-06-14 05:18:01,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:31 UTC)" (scheduled at 2024-06-14 05:18:01.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:18:01,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 18, 1, 892667), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:18:01,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:18:01,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:18:01,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:31 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:18:02,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:18:02,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:32.991899+00:00 (in 29.999438 seconds)
gcworker stdout | 2024-06-14 05:18:02,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:32 UTC)" (scheduled at 2024-06-14 05:18:02.991899+00:00)
gcworker stdout | 2024-06-14 05:18:02,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132682992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:18:03,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:18:03,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:18:03,000 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:32 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:18:03,798 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:18:03,799 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:03 UTC)" (scheduled at 2024-06-14 05:18:03.798679+00:00)
namespacegcworker stdout | 2024-06-14 05:18:03,799 [75] [DEBUG] [workers.queueworker] Getting work item from queue.
namespacegcworker stdout | 2024-06-14 05:18:03,801 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 3, 799937), True, datetime.datetime(2024, 6, 14, 5, 18, 3, 799937), 0, 'namespacegc/%', 50, 1, 0])
namespacegcworker stdout | 2024-06-14 05:18:03,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:15.800790+00:00 (in 11.999347 seconds)
namespacegcworker stdout | 2024-06-14 05:18:03,807 [75] [DEBUG] [workers.queueworker] No more work.
namespacegcworker stdout | 2024-06-14 05:18:03,807 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:18:03,807 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:03 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:18:05,378 [271] [DEBUG] [app] Starting request: urn:request:035c7251-0650-4a09-8e1a-c91b30ccf6a3 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:18:05,378 [268] [DEBUG] [app] Starting request: urn:request:f36eaf92-1f23-4784-948c-23c06c0bf472 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:18:05,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:18:05,382 [280] [DEBUG] [app] Starting request: urn:request:308487e7-31ba-4532-9d7b-c84c37b1e141 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:18:05,383 [280] [DEBUG] [app] Ending request: urn:request:308487e7-31ba-4532-9d7b-c84c37b1e141 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:308487e7-31ba-4532-9d7b-c84c37b1e141', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:18:05,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:18:05,383 [268] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080
gunicorn-web stdout | 2024-06-14 05:18:05,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:18:05,384 [280] [DEBUG] [app] Starting request: urn:request:03f6b53a-a03e-45f4-8819-fd5013cbd263 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:05,385 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:18:05,385 [280] [DEBUG] [app] Ending request: urn:request:03f6b53a-a03e-45f4-8819-fd5013cbd263 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:03f6b53a-a03e-45f4-8819-fd5013cbd263', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:18:05,385 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-web stdout | 2024-06-14 05:18:05,385 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:05,386 [271] [DEBUG] [app] Starting request: urn:request:b4201d2c-7a31-4e52-9980-a4e9546b12ea (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:05,386 [271] [DEBUG] [app] Ending request: urn:request:b4201d2c-7a31-4e52-9980-a4e9546b12ea (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b4201d2c-7a31-4e52-9980-a4e9546b12ea', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:18:05,386 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:18:05,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:18:05,387 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:05,387 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:18:05,388 [270] [DEBUG] [app] Starting request: urn:request:c18cd6af-b041-42a0-8209-09bcf3c59e64 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:05,388 [270] [DEBUG] [app] Ending request: urn:request:c18cd6af-b041-42a0-8209-09bcf3c59e64 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c18cd6af-b041-42a0-8209-09bcf3c59e64', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:18:05,389 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:18:05,389 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.001)
gunicorn-web stdout | 2024-06-14 05:18:05,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2024, 6, 14, 5, 18, 5, 390031), None, 'quay', 'quay', datetime.datetime(2024, 6, 14, 5, 18, 5, 390057), 'quay', None, datetime.datetime(2024, 6, 13, 5, 18, 5, 390067), datetime.datetime(2024, 6, 7, 5, 18, 5, 390075), None])
gunicorn-web stdout | 2024-06-14 05:18:05,391 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:18:05,395 [271] [DEBUG] [app] Ending request: urn:request:035c7251-0650-4a09-8e1a-c91b30ccf6a3 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:035c7251-0650-4a09-8e1a-c91b30ccf6a3', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:18:05,395 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:18:05,395 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:18:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:18:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:18:05,400 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:18:05,404 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:18:05,405 [268] [DEBUG] [app] Ending request: urn:request:f36eaf92-1f23-4784-948c-23c06c0bf472 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f36eaf92-1f23-4784-948c-23c06c0bf472', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:18:05,406 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:18:05,406 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:18:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:18:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.029 118 0.029)
exportactionlogsworker stdout | 2024-06-14 05:18:06,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:18:06,199 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:06 UTC)" (scheduled at 2024-06-14 05:18:06.198336+00:00)
exportactionlogsworker stdout | 2024-06-14 05:18:06,199 [66] [DEBUG] [workers.queueworker] Running watchdog.
exportactionlogsworker stdout | 2024-06-14 05:18:06,199 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:06 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:18:06,200 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:23.197203+00:00 (in 16.997163 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:18:07,198 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:18:07,199 [87] [INFO] [apscheduler.executors.default] Running job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:07 UTC)" (scheduled at 2024-06-14 05:18:07.198655+00:00)
quotaregistrysizeworker stdout | 2024-06-14 05:18:07,200 [87] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."size_bytes", "t1"."running", "t1"."queued", "t1"."completed_ms" FROM "quotaregistrysize" AS "t1" LIMIT %s OFFSET %s', [1, 0])
quotaregistrysizeworker stdout | 2024-06-14 05:18:07,200 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:07.198655+00:00 (in 59.997686 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:18:07,206 [87] [DEBUG] [data.database] Disconnecting from database.
quotaregistrysizeworker stdout | 2024-06-14 05:18:07,206 [87] [INFO] [apscheduler.executors.default] Job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:07 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:18:08,472 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:18:09,264 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:18:11,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:18:11,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:21.591084+00:00 (in 9.999412 seconds)
notificationworker stdout | 2024-06-14 05:18:11,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:21 UTC)" (scheduled at 2024-06-14 05:18:11.591084+00:00)
notificationworker stdout | 2024-06-14 05:18:11,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:18:11,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 11, 592140), True, datetime.datetime(2024, 6, 14, 5, 18, 11, 592140), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:18:11,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:18:11,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:18:11,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:21 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:18:13,973 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:18:15,242 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
namespacegcworker stdout | 2024-06-14 05:18:15,801 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:18:15,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:03.798679+00:00 (in 47.997233 seconds)
namespacegcworker stdout | 2024-06-14 05:18:15,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:15 UTC)" (scheduled at 2024-06-14 05:18:15.800790+00:00)
namespacegcworker stdout | 2024-06-14 05:18:15,801 [75] [DEBUG] [workers.queueworker] Running watchdog.
namespacegcworker stdout | 2024-06-14 05:18:15,801 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:15 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:18:15,804 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:18:16,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:18:16,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:28.194620+00:00 (in 12.002461 seconds)
securityworker stdout | 2024-06-14 05:18:16,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:46 UTC)" (scheduled at 2024-06-14 05:18:16.191483+00:00)
securityworker stdout | 2024-06-14 05:18:16,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:18:16,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:18:16,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:18:16,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:18:16,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:18:16,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:18:16,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:46 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:18:16,463 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:18:16,963 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:18:18,345 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:18:18,560 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:18:19,012 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:18:19,016 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:18:19,020 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:18:19,023 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:18:19,026 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:18:20,378 [268] [DEBUG] [app] Starting request: urn:request:809846e0-1209-4980-a027-380863e58bb1 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:18:20,378 [271] [DEBUG] [app] Starting request: urn:request:9c3cec9c-b07e-4453-b73c-77ab1beee1a2 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:18:20,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:18:20,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:18:20,382 [280] [DEBUG] [app] Starting request: urn:request:c0aab55a-e6f6-4616-ac94-71ecee66052d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:18:20,382 [287] [DEBUG] [app] Starting request: urn:request:6fe1e5cf-8be1-4d9d-afd9-910e019f07d5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:18:20,382 [280] [DEBUG] [app] Ending request: urn:request:c0aab55a-e6f6-4616-ac94-71ecee66052d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c0aab55a-e6f6-4616-ac94-71ecee66052d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:18:20,382 [287] [DEBUG] [app] Ending request: urn:request:6fe1e5cf-8be1-4d9d-afd9-910e019f07d5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6fe1e5cf-8be1-4d9d-afd9-910e019f07d5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:18:20,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:18:20,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:18:20,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:20,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:20,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:18:20,385 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:18:20,385 [271] [DEBUG] [app] Starting request: urn:request:1758c1fd-5be4-44aa-8326-ff54a5ac01d7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:20,386 [271] [DEBUG] [app] Ending request: urn:request:1758c1fd-5be4-44aa-8326-ff54a5ac01d7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1758c1fd-5be4-44aa-8326-ff54a5ac01d7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:18:20,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:18:20,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:20,387 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:18:20,387 [270] [DEBUG] [app] Starting request: urn:request:b61c31d5-703d-46dc-ba5d-2c06ea29cb20 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:20,387 [270] [DEBUG] [app] Ending request: urn:request:b61c31d5-703d-46dc-ba5d-2c06ea29cb20 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b61c31d5-703d-46dc-ba5d-2c06ea29cb20', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:18:20,387 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:18:20,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:20,388 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:18:20,391 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:18:20,392 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:18:20,395 [271] [DEBUG] [app] Ending request: urn:request:9c3cec9c-b07e-4453-b73c-77ab1beee1a2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9c3cec9c-b07e-4453-b73c-77ab1beee1a2', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:18:20,395 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:18:20,395 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:18:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:18:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.017 118 0.018)
gunicorn-web stdout | 2024-06-14 05:18:20,396 [268] [DEBUG] [app] Ending request: urn:request:809846e0-1209-4980-a027-380863e58bb1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:809846e0-1209-4980-a027-380863e58bb1', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:18:20,397 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:18:20,397 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:18:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:18:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.020)
notificationworker stdout | 2024-06-14 05:18:21,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:18:21,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:21.592513+00:00 (in 0.000825 seconds)
notificationworker stdout | 2024-06-14 05:18:21,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:31 UTC)" (scheduled at 2024-06-14 05:18:21.591084+00:00)
notificationworker stdout | 2024-06-14 05:18:21,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:18:21,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 21, 592148), True, datetime.datetime(2024, 6, 14, 5, 18, 21, 592148), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:18:21,593 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:18:21,593 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:21 UTC)" (scheduled at 2024-06-14 05:18:21.592513+00:00)
notificationworker stdout | 2024-06-14 05:18:21,593 [82] [DEBUG] [workers.queueworker] Running watchdog.
notificationworker stdout | 2024-06-14 05:18:21,593 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:18:21 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:18:21,594 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:31.591084+00:00 (in 9.997054 seconds)
notificationworker stdout | 2024-06-14 05:18:21,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:18:21,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:18:21,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:31 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:18:21,850 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:18:22,261 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:18:22,633 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:18:22,721 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
exportactionlogsworker stdout | 2024-06-14 05:18:23,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:18:23,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:06.198336+00:00 (in 43.000535 seconds)
exportactionlogsworker stdout | 2024-06-14 05:18:23,197 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:23 UTC)" (scheduled at 2024-06-14 05:18:23.197203+00:00)
exportactionlogsworker stdout | 2024-06-14 05:18:23,198 [66] [DEBUG] [workers.queueworker] Getting work item from queue.
exportactionlogsworker stdout | 2024-06-14 05:18:23,199 [66] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 23, 198219), True, datetime.datetime(2024, 6, 14, 5, 18, 23, 198219), 0, 'exportactionlogs/%', 50, 1, 0])
exportactionlogsworker stdout | 2024-06-14 05:18:23,206 [66] [DEBUG] [workers.queueworker] No more work.
exportactionlogsworker stdout | 2024-06-14 05:18:23,206 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:18:23,206 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:23 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:18:24,850 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:18:25,140 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:18:25,540 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:18:25,738 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
autopruneworker stdout | 2024-06-14 05:18:25,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:18:25,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:55.804090+00:00 (in 29.999316 seconds)
autopruneworker stdout | 2024-06-14 05:18:25,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:55 UTC)" (scheduled at 2024-06-14 05:18:25.804090+00:00)
autopruneworker stdout | 2024-06-14 05:18:25,810 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342245809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:18:25,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:18:25,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:18:25,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:55 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:18:26,939 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:18:27,559 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:18:27,624 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:18:27,972 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:18:28,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:18:28,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:46.191483+00:00 (in 17.996212 seconds)
securityworker stdout | 2024-06-14 05:18:28,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:58 UTC)" (scheduled at 2024-06-14 05:18:28.194620+00:00)
securityworker stdout | 2024-06-14 05:18:28,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:18:28,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:18:28,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:18:28,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:18:28,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:18:28,198 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:18:28,198 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:18:28,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:18:28,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:18:28,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:18:28,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:18:28,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:18:28,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 13, 28, 203783), 1, 2])
securityworker stdout | 2024-06-14 05:18:28,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:18:28,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:18:28,213 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 13, 28, 203783), 1, 2])
securityworker stdout | 2024-06-14 05:18:28,213 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:18:28,213 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:18:28,214 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:18:28,214 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:18:28,214 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:18:28,214 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:18:28,214 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:18:28,214 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:18:28,214 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:18:28,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:18:58 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:18:28,962 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:18:29,347 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:18:29,350 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:18:29,354 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:18:30,211 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:18:30,214 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:18:30,218 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:18:30,222 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:18:30,225 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:18:30,228 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:18:30,231 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:18:30,234 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
gunicorn-registry stdout | 2024-06-14 05:18:30,237 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
securityscanningnotificationworker stdout | 2024-06-14 05:18:31,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:18:31,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:35.332183+00:00 (in 4.000351 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:18:31,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:31 UTC)" (scheduled at 2024-06-14 05:18:31.331172+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:18:31,332 [110] [DEBUG] [workers.queueworker] Getting work item from queue.
securityscanningnotificationworker stdout | 2024-06-14 05:18:31,333 [110] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 31, 332282), True, datetime.datetime(2024, 6, 14, 5, 18, 31, 332282), 0, 'secscanv4/%', 50, 1, 0])
securityscanningnotificationworker stdout | 2024-06-14 05:18:31,339 [110] [DEBUG] [workers.queueworker] No more work.
securityscanningnotificationworker stdout | 2024-06-14 05:18:31,339 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:18:31,339 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:31 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:18:31,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:18:31,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:41.591084+00:00 (in 9.999433 seconds)
notificationworker stdout | 2024-06-14 05:18:31,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:41 UTC)" (scheduled at 2024-06-14 05:18:31.591084+00:00)
notificationworker stdout | 2024-06-14 05:18:31,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:18:31,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 31, 592141), True, datetime.datetime(2024, 6, 14, 5, 18, 31, 592141), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:18:31,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:18:31,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:18:31,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:41 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:18:31,891 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:18:31,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:01.891735+00:00 (in 29.999431 seconds)
buildlogsarchiver stdout | 2024-06-14 05:18:31,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:01 UTC)" (scheduled at 2024-06-14 05:18:31.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:18:31,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 18, 31, 892711), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:18:31,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:18:31,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:18:31,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:01 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:18:32,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:18:32,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:02.991899+00:00 (in 29.999497 seconds)
gcworker stdout | 2024-06-14 05:18:32,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:02 UTC)" (scheduled at 2024-06-14 05:18:32.991899+00:00)
gcworker stdout | 2024-06-14 05:18:32,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132712992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:18:33,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:18:33,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:18:33,000 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:02 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:18:34,997 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:18:34,997 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:42.089819+00:00 (in 7.092298 seconds)
repositorygcworker stdout | 2024-06-14 05:18:34,997 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:34 UTC)" (scheduled at 2024-06-14 05:18:34.996948+00:00)
repositorygcworker stdout | 2024-06-14 05:18:34,997 [109] [DEBUG] [workers.queueworker] Getting work item from queue.
repositorygcworker stdout | 2024-06-14 05:18:34,999 [109] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 34, 997985), True, datetime.datetime(2024, 6, 14, 5, 18, 34, 997985), 0, 'repositorygc/%', 50, 1, 0])
repositorygcworker stdout | 2024-06-14 05:18:35,005 [109] [DEBUG] [workers.queueworker] No more work.
repositorygcworker stdout | 2024-06-14 05:18:35,005 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:18:35,006 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:34 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:18:35,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:18:35,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:31.331172+00:00 (in 55.998381 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:18:35,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:35 UTC)" (scheduled at 2024-06-14 05:18:35.332183+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:18:35,333 [110] [DEBUG] [workers.queueworker] Running watchdog.
securityscanningnotificationworker stdout | 2024-06-14 05:18:35,333 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:35 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:18:35,378 [270] [DEBUG] [app] Starting request: urn:request:4210e15f-2f65-4d73-b64f-2b766f835e92 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:18:35,380 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:18:35,381 [269] [DEBUG] [app] Starting request: urn:request:d4bc2974-55f4-447d-9438-7611b9a09b66 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-registry stdout | 2024-06-14 05:18:35,382 [280] [DEBUG] [app] Starting request: urn:request:40d9b12c-4742-4e5f-9afa-07d240ee90d2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:18:35,383 [280] [DEBUG] [app] Ending request: urn:request:40d9b12c-4742-4e5f-9afa-07d240ee90d2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:40d9b12c-4742-4e5f-9afa-07d240ee90d2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:18:35,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.002)
gunicorn-web stdout | 2024-06-14 05:18:35,383 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:35,385 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:18:35,385 [269] [DEBUG] [urllib3.connectionpool] Starting new HTTP connection (1): localhost:8080
gunicorn-web stdout | 2024-06-14 05:18:35,386 [271] [DEBUG] [app] Starting request: urn:request:fdcaba7e-779e-41f9-a78c-24f81defb1e7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:35,387 [271] [DEBUG] [app] Ending request: urn:request:fdcaba7e-779e-41f9-a78c-24f81defb1e7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:fdcaba7e-779e-41f9-a78c-24f81defb1e7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:18:35,387 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:18:35,387 [287] [DEBUG] [app] Starting request: urn:request:970d4365-b09b-4cc5-af25-7a65b441dcc2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:35,387 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:18:35,387 [287] [DEBUG] [app] Ending request: urn:request:970d4365-b09b-4cc5-af25-7a65b441dcc2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:970d4365-b09b-4cc5-af25-7a65b441dcc2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:18:35,388 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:18:35,388 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:18:35,388 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:35,389 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:18:35,390 [271] [DEBUG] [app] Starting request: urn:request:abad6868-c1ec-46e8-8ff4-c76d8f876656 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:35,391 [271] [DEBUG] [app] Ending request: urn:request:abad6868-c1ec-46e8-8ff4-c76d8f876656 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:abad6868-c1ec-46e8-8ff4-c76d8f876656', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:18:35,392 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:18:35,392 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:18:35,392 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:18:35,394 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name", "t1"."kid", "t1"."service", "t1"."jwk", "t1"."metadata", "t1"."created_date", "t1"."expiration_date", "t1"."rotation_duration", "t1"."approval_id" FROM "servicekey" AS "t1" LEFT OUTER JOIN "servicekeyapproval" AS "t2" ON ("t1"."approval_id" = "t2"."id") WHERE ((((NOT ("t1"."approval_id" IS %s) AND (("t1"."expiration_date" > %s) OR ("t1"."expiration_date" IS %s))) AND ("t1"."service" = %s)) AND (NOT (("t1"."service" = %s) AND ("t1"."expiration_date" <= %s)) OR NOT ((("t1"."service" = %s) AND ("t1"."approval_id" IS %s)) AND ("t1"."created_date" <= %s)))) AND (NOT ("t1"."expiration_date" <= %s) OR ("t1"."expiration_date" IS %s)))', [None, datetime.datetime(2024, 6, 14, 5, 18, 35, 393572), None, 'quay', 'quay', datetime.datetime(2024, 6, 14, 5, 18, 35, 393597), 'quay', None, datetime.datetime(2024, 6, 13, 5, 18, 35, 393614), datetime.datetime(2024, 6, 7, 5, 18, 35, 393625), None])
gunicorn-web stdout | 2024-06-14 05:18:35,396 [270] [DEBUG] [app] Ending request: urn:request:4210e15f-2f65-4d73-b64f-2b766f835e92 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4210e15f-2f65-4d73-b64f-2b766f835e92', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:18:35,397 [270] [DEBUG] [data.database] Disconnecting from database.
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:18:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.020)
gunicorn-web stdout | 2024-06-14 05:18:35,397 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:18:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:18:35,402 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:18:35,406 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:18:35,407 [269] [DEBUG] [app] Ending request: urn:request:d4bc2974-55f4-447d-9438-7611b9a09b66 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d4bc2974-55f4-447d-9438-7611b9a09b66', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:18:35,407 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:18:35,408 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:18:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:18:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.030 118 0.030)
chunkcleanupworker stdout | 2024-06-14 05:18:38,508 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:18:39,271 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:18:41,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:18:41,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:51.591084+00:00 (in 9.999469 seconds)
notificationworker stdout | 2024-06-14 05:18:41,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:51 UTC)" (scheduled at 2024-06-14 05:18:41.591084+00:00)
notificationworker stdout | 2024-06-14 05:18:41,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:18:41,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 41, 592043), True, datetime.datetime(2024, 6, 14, 5, 18, 41, 592043), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:18:41,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:18:41,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:18:41,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:18:51 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:18:42,090 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:18:42,090 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:34.996948+00:00 (in 52.906628 seconds)
repositorygcworker stdout | 2024-06-14 05:18:42,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:42 UTC)" (scheduled at 2024-06-14 05:18:42.089819+00:00)
repositorygcworker stdout | 2024-06-14 05:18:42,090 [109] [DEBUG] [workers.queueworker] Running watchdog.
repositorygcworker stdout | 2024-06-14 05:18:42,090 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:19:42 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:18:44,009 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:18:45,250 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
repositorygcworker stdout | 2024-06-14 05:18:45,831 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:18:46,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:18:46,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:18:58.194620+00:00 (in 12.002562 seconds)
securityworker stdout | 2024-06-14 05:18:46,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:16 UTC)" (scheduled at 2024-06-14 05:18:46.191483+00:00)
securityworker stdout | 2024-06-14 05:18:46,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:18:46,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:18:46,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:18:46,196 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:18:46,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:18:46,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:18:46,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:16 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:18:46,501 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:18:46,993 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:18:48,382 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:18:48,579 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:18:49,022 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:18:49,025 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:18:49,029 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:18:49,031 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:18:49,034 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:18:50,378 [271] [DEBUG] [app] Starting request: urn:request:ef89f7e3-7b9f-4e24-881c-84e161e7ba67 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:18:50,378 [269] [DEBUG] [app] Starting request: urn:request:2f524b10-5246-4227-9617-4038170c9568 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:18:50,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:18:50,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:18:50,381 [287] [DEBUG] [app] Starting request: urn:request:bdc28208-7ddd-454c-adb0-c22927e2b6d1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:18:50,382 [287] [DEBUG] [app] Ending request: urn:request:bdc28208-7ddd-454c-adb0-c22927e2b6d1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:bdc28208-7ddd-454c-adb0-c22927e2b6d1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:18:50,382 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:18:50,382 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:18:50,383 [286] [DEBUG] [app] Starting request: urn:request:98fb838b-93b4-46bb-961f-d91701f8d10d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:50,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:18:50,384 [286] [DEBUG] [app] Ending request: urn:request:98fb838b-93b4-46bb-961f-d91701f8d10d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:98fb838b-93b4-46bb-961f-d91701f8d10d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.004 162 0.003)
gunicorn-registry stdout | 2024-06-14 05:18:50,385 [286] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:18:50,385 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:50,385 [271] [DEBUG] [app] Starting request: urn:request:711dccfb-15f5-4f19-9ad6-df8a89be4ed9 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:50,385 [271] [DEBUG] [app] Ending request: urn:request:711dccfb-15f5-4f19-9ad6-df8a89be4ed9 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:711dccfb-15f5-4f19-9ad6-df8a89be4ed9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:18:50,385 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:18:50,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:50,386 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:18:50,386 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:18:50,387 [269] [DEBUG] [app] Starting request: urn:request:82f0fec0-48cc-47dc-b810-082fe3903e49 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:18:50,388 [269] [DEBUG] [app] Ending request: urn:request:82f0fec0-48cc-47dc-b810-082fe3903e49 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:82f0fec0-48cc-47dc-b810-082fe3903e49', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:18:50,388 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:18:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:18:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:18:50,388 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:18:50,389 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:18:50,390 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:18:50,393 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:18:50,394 [271] [DEBUG] [app] Ending request: urn:request:ef89f7e3-7b9f-4e24-881c-84e161e7ba67 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ef89f7e3-7b9f-4e24-881c-84e161e7ba67', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:18:50,394 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:18:50,394 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:18:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:18:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.017 118 0.017)
gunicorn-web stdout | 2024-06-14 05:18:50,397 [269] [DEBUG] [app] Ending request: urn:request:2f524b10-5246-4227-9617-4038170c9568 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2f524b10-5246-4227-9617-4038170c9568', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:18:50,397 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:18:50,397 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:18:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:18:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
notificationworker stdout | 2024-06-14 05:18:51,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:18:51,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:01.591084+00:00 (in 9.999464 seconds)
notificationworker stdout | 2024-06-14 05:18:51,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:01 UTC)" (scheduled at 2024-06-14 05:18:51.591084+00:00)
notificationworker stdout | 2024-06-14 05:18:51,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:18:51,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 18, 51, 592084), True, datetime.datetime(2024, 6, 14, 5, 18, 51, 592084), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:18:51,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:18:51,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:18:51,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:01 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:18:51,886 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:18:52,298 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:18:52,648 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:18:52,757 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
quotaregistrysizeworker stdout | 2024-06-14 05:18:54,886 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:18:55,173 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:18:55,548 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:18:55,750 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
autopruneworker stdout | 2024-06-14 05:18:55,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:18:55,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:25.804090+00:00 (in 29.999323 seconds)
autopruneworker stdout | 2024-06-14 05:18:55,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:25 UTC)" (scheduled at 2024-06-14 05:18:55.804090+00:00)
autopruneworker stdout | 2024-06-14 05:18:55,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342275809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:18:55,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:18:55,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:18:55,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:25 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:18:56,962 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:18:57,597 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:18:57,660 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:18:57,991 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:18:58,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:18:58,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:16.191483+00:00 (in 17.996319 seconds)
securityworker stdout | 2024-06-14 05:18:58,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:28 UTC)" (scheduled at 2024-06-14 05:18:58.194620+00:00)
securityworker stdout | 2024-06-14 05:18:58,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:18:58,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:18:58,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:18:58,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:18:58,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:18:58,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:18:58,198 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:18:58,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:18:58,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:18:58,210 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:18:58,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:18:58,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:18:58,212 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 13, 58, 204604), 1, 2])
securityworker stdout | 2024-06-14 05:18:58,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:18:58,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:18:58,213 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 13, 58, 204604), 1, 2])
securityworker stdout | 2024-06-14 05:18:58,214 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:18:58,214 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:18:58,214 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:18:58,214 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:18:58,214 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:18:58,214 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:18:58,214 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:18:58,214 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:18:58,214 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:18:58,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:28 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:18:58,977 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:18:59,356 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:18:59,359 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:18:59,362 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:19:00,223 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:19:00,226 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:19:00,228 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:19:00,234 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:19:00,237 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:19:00,242 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:19:00,245 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:19:00,247 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
gunicorn-registry stdout | 2024-06-14 05:19:00,249 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
notificationworker stdout | 2024-06-14 05:19:01,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:19:01,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:11.591084+00:00 (in 9.999437 seconds)
notificationworker stdout | 2024-06-14 05:19:01,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:11 UTC)" (scheduled at 2024-06-14 05:19:01.591084+00:00)
notificationworker stdout | 2024-06-14 05:19:01,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:19:01,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 1, 592157), True, datetime.datetime(2024, 6, 14, 5, 19, 1, 592157), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:19:01,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:19:01,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:19:01,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:11 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:19:01,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:19:01,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:31.891735+00:00 (in 29.999279 seconds)
buildlogsarchiver stdout | 2024-06-14 05:19:01,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:31 UTC)" (scheduled at 2024-06-14 05:19:01.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:19:01,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 19, 1, 892886), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:19:01,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:19:01,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:19:01,900 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:31 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:19:02,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:19:02,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:32.991899+00:00 (in 29.999432 seconds)
gcworker stdout | 2024-06-14 05:19:02,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:32 UTC)" (scheduled at 2024-06-14 05:19:02.991899+00:00)
gcworker stdout | 2024-06-14 05:19:02,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132742992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:19:03,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:19:03,001 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:19:03,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:32 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:19:03,798 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:19:03,799 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:15.800790+00:00 (in 12.001369 seconds)
namespacegcworker stdout | 2024-06-14 05:19:03,799 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:03 UTC)" (scheduled at 2024-06-14 05:19:03.798679+00:00)
namespacegcworker stdout | 2024-06-14 05:19:03,799 [75] [DEBUG] [workers.queueworker] Getting work item from queue.
namespacegcworker stdout | 2024-06-14 05:19:03,800 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 3, 799876), True, datetime.datetime(2024, 6, 14, 5, 19, 3, 799876), 0, 'namespacegc/%', 50, 1, 0])
namespacegcworker stdout | 2024-06-14 05:19:03,806 [75] [DEBUG] [workers.queueworker] No more work.
namespacegcworker stdout | 2024-06-14 05:19:03,806 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:19:03,806 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:03 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:19:05,379 [268] [DEBUG] [app] Starting request: urn:request:6b1baad4-dfe9-4e30-b2a4-75574bef8b1b (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:19:05,379 [269] [DEBUG] [app] Starting request: urn:request:4c00b8b9-9edc-43fc-99e7-c0f873906244 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:19:05,381 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:05,381 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:19:05,384 [285] [DEBUG] [app] Starting request: urn:request:163ec2df-6ccb-43b4-9aa6-f0780dd22cdb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:19:05,384 [281] [DEBUG] [app] Starting request: urn:request:0522e354-5f7e-4614-babd-abf9d405635d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:19:05,385 [281] [DEBUG] [app] Ending request: urn:request:0522e354-5f7e-4614-babd-abf9d405635d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:0522e354-5f7e-4614-babd-abf9d405635d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:19:05,385 [285] [DEBUG] [app] Ending request: urn:request:163ec2df-6ccb-43b4-9aa6-f0780dd22cdb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:163ec2df-6ccb-43b4-9aa6-f0780dd22cdb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.003 162 0.004)
gunicorn-registry stdout | 2024-06-14 05:19:05,385 [281] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:19:05,385 [285] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:19:05,385 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.003 162 0.004)
gunicorn-web stdout | 2024-06-14 05:19:05,386 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:05,387 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:05,387 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:05,388 [271] [DEBUG] [app] Starting request: urn:request:5c0339bb-3337-429e-ad76-def58f1ae8a6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:05,389 [268] [DEBUG] [app] Starting request: urn:request:dba57ea0-d83d-4405-962e-f0f117ad71ef (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:05,389 [268] [DEBUG] [app] Ending request: urn:request:dba57ea0-d83d-4405-962e-f0f117ad71ef (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:dba57ea0-d83d-4405-962e-f0f117ad71ef', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:19:05,389 [271] [DEBUG] [app] Ending request: urn:request:5c0339bb-3337-429e-ad76-def58f1ae8a6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5c0339bb-3337-429e-ad76-def58f1ae8a6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:19:05,389 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:19:05,389 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.002)
gunicorn-web stdout | 2024-06-14 05:19:05,389 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:05,389 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:05,390 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:19:05,390 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:19:05,394 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:19:05,394 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:19:05,398 [268] [DEBUG] [app] Ending request: urn:request:6b1baad4-dfe9-4e30-b2a4-75574bef8b1b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6b1baad4-dfe9-4e30-b2a4-75574bef8b1b', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:19:05,398 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:19:05,399 [269] [DEBUG] [app] Ending request: urn:request:4c00b8b9-9edc-43fc-99e7-c0f873906244 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4c00b8b9-9edc-43fc-99e7-c0f873906244', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:19:05,399 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:19:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:19:05,399 [269] [DEBUG] [data.database] Disconnecting from database.
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:19:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.021 118 0.020)
gunicorn-web stdout | 2024-06-14 05:19:05,399 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:19:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:19:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.021 118 0.021)
exportactionlogsworker stdout | 2024-06-14 05:19:06,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:19:06,198 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:23.197203+00:00 (in 16.998238 seconds)
exportactionlogsworker stdout | 2024-06-14 05:19:06,199 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:06 UTC)" (scheduled at 2024-06-14 05:19:06.198336+00:00)
exportactionlogsworker stdout | 2024-06-14 05:19:06,199 [66] [DEBUG] [workers.queueworker] Running watchdog.
exportactionlogsworker stdout | 2024-06-14 05:19:06,199 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:06 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:19:07,198 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:19:07,199 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:07.198655+00:00 (in 59.999435 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:19:07,199 [87] [INFO] [apscheduler.executors.default] Running job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:07 UTC)" (scheduled at 2024-06-14 05:19:07.198655+00:00)
quotaregistrysizeworker stdout | 2024-06-14 05:19:07,200 [87] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."size_bytes", "t1"."running", "t1"."queued", "t1"."completed_ms" FROM "quotaregistrysize" AS "t1" LIMIT %s OFFSET %s', [1, 0])
quotaregistrysizeworker stdout | 2024-06-14 05:19:07,205 [87] [DEBUG] [data.database] Disconnecting from database.
quotaregistrysizeworker stdout | 2024-06-14 05:19:07,205 [87] [INFO] [apscheduler.executors.default] Job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:07 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:19:08,544 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:19:09,289 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:19:11,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:19:11,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:21.591084+00:00 (in 9.999369 seconds)
notificationworker stdout | 2024-06-14 05:19:11,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:21 UTC)" (scheduled at 2024-06-14 05:19:11.591084+00:00)
notificationworker stdout | 2024-06-14 05:19:11,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:19:11,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 11, 592199), True, datetime.datetime(2024, 6, 14, 5, 19, 11, 592199), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:19:11,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:19:11,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:19:11,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:21 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:19:14,024 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:19:15,258 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
namespacegcworker stdout | 2024-06-14 05:19:15,801 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:19:15,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:03.798679+00:00 (in 47.997174 seconds)
namespacegcworker stdout | 2024-06-14 05:19:15,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:15 UTC)" (scheduled at 2024-06-14 05:19:15.800790+00:00)
namespacegcworker stdout | 2024-06-14 05:19:15,801 [75] [DEBUG] [workers.queueworker] Running watchdog.
namespacegcworker stdout | 2024-06-14 05:19:15,801 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:15 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:19:15,867 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:19:16,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:19:16,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:28.194620+00:00 (in 12.002627 seconds)
securityworker stdout | 2024-06-14 05:19:16,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:46 UTC)" (scheduled at 2024-06-14 05:19:16.191483+00:00)
securityworker stdout | 2024-06-14 05:19:16,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:19:16,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:19:16,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:19:16,196 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:19:16,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:19:16,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:19:16,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:46 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:19:16,515 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:19:17,030 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:19:18,395 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:19:18,606 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:19:19,031 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:19:19,035 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:19:19,037 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:19:19,040 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:19:19,043 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:19:20,379 [268] [DEBUG] [app] Starting request: urn:request:2033b9da-ec69-487c-9d34-cc11382b74b1 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:19:20,379 [271] [DEBUG] [app] Starting request: urn:request:d9ebf108-3719-47c5-af18-7b458211dff7 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:19:20,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:20,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:19:20,383 [280] [DEBUG] [app] Starting request: urn:request:fa42b002-c961-48db-883f-57e397a138ce (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:19:20,383 [280] [DEBUG] [app] Ending request: urn:request:fa42b002-c961-48db-883f-57e397a138ce (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:fa42b002-c961-48db-883f-57e397a138ce', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.003 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:19:20,384 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:19:20,384 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:19:20,384 [283] [DEBUG] [app] Starting request: urn:request:6f7cd0f8-8053-4e20-ad72-a49876aa663f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:20,385 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:19:20,385 [283] [DEBUG] [app] Ending request: urn:request:6f7cd0f8-8053-4e20-ad72-a49876aa663f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:6f7cd0f8-8053-4e20-ad72-a49876aa663f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.005 162 0.005)
gunicorn-web stdout | 2024-06-14 05:19:20,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:19:20,386 [283] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:19:20,386 [268] [DEBUG] [app] Starting request: urn:request:9d19010f-9b3a-4c5e-a9e5-d536da6a010a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:20,386 [268] [DEBUG] [app] Ending request: urn:request:9d19010f-9b3a-4c5e-a9e5-d536da6a010a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9d19010f-9b3a-4c5e-a9e5-d536da6a010a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.000)
gunicorn-web stdout | 2024-06-14 05:19:20,387 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:19:20,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:20,387 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:20,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:19:20,388 [271] [DEBUG] [app] Starting request: urn:request:36e932cc-ec6f-485a-a2fa-0953e6d0b189 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:20,388 [271] [DEBUG] [app] Ending request: urn:request:36e932cc-ec6f-485a-a2fa-0953e6d0b189 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:36e932cc-ec6f-485a-a2fa-0953e6d0b189', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:19:20,389 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:19:20,389 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:20,389 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:19:20,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:19:20,393 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:19:20,396 [268] [DEBUG] [app] Ending request: urn:request:2033b9da-ec69-487c-9d34-cc11382b74b1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2033b9da-ec69-487c-9d34-cc11382b74b1', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:19:20,396 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:19:20,397 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:19:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:19:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.018)
gunicorn-web stdout | 2024-06-14 05:19:20,397 [271] [DEBUG] [app] Ending request: urn:request:d9ebf108-3719-47c5-af18-7b458211dff7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d9ebf108-3719-47c5-af18-7b458211dff7', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:19:20,397 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:19:20,398 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:19:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:19:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.019)
notificationworker stdout | 2024-06-14 05:19:21,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:19:21,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:21.592513+00:00 (in 0.000761 seconds)
notificationworker stdout | 2024-06-14 05:19:21,592 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:31 UTC)" (scheduled at 2024-06-14 05:19:21.591084+00:00)
notificationworker stdout | 2024-06-14 05:19:21,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:19:21,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 21, 592315), True, datetime.datetime(2024, 6, 14, 5, 19, 21, 592315), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:19:21,593 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:19:21,594 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:31.591084+00:00 (in 9.997083 seconds)
notificationworker stdout | 2024-06-14 05:19:21,594 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:21 UTC)" (scheduled at 2024-06-14 05:19:21.592513+00:00)
notificationworker stdout | 2024-06-14 05:19:21,594 [82] [DEBUG] [workers.queueworker] Running watchdog.
notificationworker stdout | 2024-06-14 05:19:21,594 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:21 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:19:21,600 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:19:21,600 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:19:21,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:31 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:19:21,916 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:19:22,320 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:19:22,686 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:19:22,785 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
exportactionlogsworker stdout | 2024-06-14 05:19:23,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:19:23,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:06.198336+00:00 (in 43.000498 seconds)
exportactionlogsworker stdout | 2024-06-14 05:19:23,198 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:23 UTC)" (scheduled at 2024-06-14 05:19:23.197203+00:00)
exportactionlogsworker stdout | 2024-06-14 05:19:23,198 [66] [DEBUG] [workers.queueworker] Getting work item from queue.
exportactionlogsworker stdout | 2024-06-14 05:19:23,199 [66] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 23, 198238), True, datetime.datetime(2024, 6, 14, 5, 19, 23, 198238), 0, 'exportactionlogs/%', 50, 1, 0])
exportactionlogsworker stdout | 2024-06-14 05:19:23,205 [66] [DEBUG] [workers.queueworker] No more work.
exportactionlogsworker stdout | 2024-06-14 05:19:23,205 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:19:23,205 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:23 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:19:24,923 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:19:25,210 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:19:25,578 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:19:25,781 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
autopruneworker stdout | 2024-06-14 05:19:25,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:19:25,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:55.804090+00:00 (in 29.999327 seconds)
autopruneworker stdout | 2024-06-14 05:19:25,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:55 UTC)" (scheduled at 2024-06-14 05:19:25.804090+00:00)
autopruneworker stdout | 2024-06-14 05:19:25,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342305808, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:19:25,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:19:25,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:19:25,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:55 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:19:26,972 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:19:27,634 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:19:27,697 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:19:28,025 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:19:28,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:19:28,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:46.191483+00:00 (in 17.996318 seconds)
securityworker stdout | 2024-06-14 05:19:28,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:58 UTC)" (scheduled at 2024-06-14 05:19:28.194620+00:00)
securityworker stdout | 2024-06-14 05:19:28,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:19:28,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:19:28,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:19:28,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:19:28,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:19:28,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:19:28,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:19:28,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:19:28,268 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:19:28,273 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:19:28,274 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:19:28,274 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:19:28,275 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 14, 28, 268056), 1, 2])
securityworker stdout | 2024-06-14 05:19:28,276 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:19:28,276 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:19:28,276 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 14, 28, 268056), 1, 2])
securityworker stdout | 2024-06-14 05:19:28,277 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:19:28,277 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:19:28,277 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:19:28,277 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:19:28,277 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:19:28,277 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:19:28,277 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:19:28,278 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:19:28,278 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:19:28,278 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:19:58 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:19:28,999 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:19:29,364 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:19:29,368 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:19:29,372 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:19:30,237 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:19:30,239 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:19:30,242 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:19:30,245 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:19:30,250 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:19:30,252 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:19:30,257 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:19:30,260 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:19:30,263 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
securityscanningnotificationworker stdout | 2024-06-14 05:19:31,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:19:31,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:35.332183+00:00 (in 4.000401 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:19:31,331 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:31 UTC)" (scheduled at 2024-06-14 05:19:31.331172+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:19:31,332 [110] [DEBUG] [workers.queueworker] Getting work item from queue.
securityscanningnotificationworker stdout | 2024-06-14 05:19:31,333 [110] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 31, 332185), True, datetime.datetime(2024, 6, 14, 5, 19, 31, 332185), 0, 'secscanv4/%', 50, 1, 0])
securityscanningnotificationworker stdout | 2024-06-14 05:19:31,339 [110] [DEBUG] [workers.queueworker] No more work.
securityscanningnotificationworker stdout | 2024-06-14 05:19:31,339 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:19:31,339 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:31 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:19:31,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:19:31,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:36.592214+00:00 (in 5.000625 seconds)
notificationworker stdout | 2024-06-14 05:19:31,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:41 UTC)" (scheduled at 2024-06-14 05:19:31.591084+00:00)
notificationworker stdout | 2024-06-14 05:19:31,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:19:31,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 31, 591879), True, datetime.datetime(2024, 6, 14, 5, 19, 31, 591879), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:19:31,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:19:31,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:19:31,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:41 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:19:31,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:19:31,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:01.891735+00:00 (in 29.999286 seconds)
buildlogsarchiver stdout | 2024-06-14 05:19:31,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:01 UTC)" (scheduled at 2024-06-14 05:19:31.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:19:31,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 19, 31, 892786), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:19:31,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:19:31,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:19:31,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:01 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:19:32,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:19:32,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:02.991899+00:00 (in 29.999391 seconds)
gcworker stdout | 2024-06-14 05:19:32,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:02 UTC)" (scheduled at 2024-06-14 05:19:32.991899+00:00)
gcworker stdout | 2024-06-14 05:19:32,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132772992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:19:33,001 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:19:33,001 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:19:33,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:02 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:19:34,997 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:19:34,997 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:42.089819+00:00 (in 7.092258 seconds)
repositorygcworker stdout | 2024-06-14 05:19:34,997 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:34 UTC)" (scheduled at 2024-06-14 05:19:34.996948+00:00)
repositorygcworker stdout | 2024-06-14 05:19:34,997 [109] [DEBUG] [workers.queueworker] Getting work item from queue.
repositorygcworker stdout | 2024-06-14 05:19:34,998 [109] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 34, 997872), True, datetime.datetime(2024, 6, 14, 5, 19, 34, 997872), 0, 'repositorygc/%', 50, 1, 0])
repositorygcworker stdout | 2024-06-14 05:19:35,004 [109] [DEBUG] [workers.queueworker] No more work.
repositorygcworker stdout | 2024-06-14 05:19:35,004 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:19:35,004 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:34 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:19:35,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:19:35,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:31.331172+00:00 (in 55.998381 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:19:35,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:35 UTC)" (scheduled at 2024-06-14 05:19:35.332183+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:19:35,333 [110] [DEBUG] [workers.queueworker] Running watchdog.
securityscanningnotificationworker stdout | 2024-06-14 05:19:35,333 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:35 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:19:35,378 [269] [DEBUG] [app] Starting request: urn:request:778d7970-d42f-4ba1-927a-39549d100c95 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:19:35,378 [270] [DEBUG] [app] Starting request: urn:request:ceea71fb-4089-4dcb-a328-647733b76fe8 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:19:35,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:35,380 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:19:35,382 [280] [DEBUG] [app] Starting request: urn:request:431f448d-b1b3-4ba4-91c8-e8ab35e190d8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:19:35,382 [280] [DEBUG] [app] Ending request: urn:request:431f448d-b1b3-4ba4-91c8-e8ab35e190d8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:431f448d-b1b3-4ba4-91c8-e8ab35e190d8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:19:35,382 [286] [DEBUG] [app] Starting request: urn:request:71d013e7-db3d-426d-9ef9-1f77fac81a04 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:19:35,382 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-web stdout | 2024-06-14 05:19:35,383 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:19:35,383 [286] [DEBUG] [app] Ending request: urn:request:71d013e7-db3d-426d-9ef9-1f77fac81a04 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:71d013e7-db3d-426d-9ef9-1f77fac81a04', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:19:35,383 [286] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:19:35,383 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:35,384 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:35,385 [269] [DEBUG] [app] Starting request: urn:request:9bf05f20-c814-4b23-a5a8-297bb379fc5f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:35,385 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:35,385 [269] [DEBUG] [app] Ending request: urn:request:9bf05f20-c814-4b23-a5a8-297bb379fc5f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9bf05f20-c814-4b23-a5a8-297bb379fc5f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:19:35,386 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:19:35,386 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:35,386 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:19:35,387 [271] [DEBUG] [app] Starting request: urn:request:d174bd1b-1beb-48eb-9cb7-9403158444b8 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:35,387 [271] [DEBUG] [app] Ending request: urn:request:d174bd1b-1beb-48eb-9cb7-9403158444b8 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d174bd1b-1beb-48eb-9cb7-9403158444b8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:19:35,388 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:19:35,388 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:35,389 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:19:35,390 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:19:35,393 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:19:35,395 [269] [DEBUG] [app] Ending request: urn:request:778d7970-d42f-4ba1-927a-39549d100c95 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:778d7970-d42f-4ba1-927a-39549d100c95', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:19:35,395 [269] [DEBUG] [data.database] Disconnecting from database.
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:19:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:19:35,395 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:19:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:19:35,397 [270] [DEBUG] [app] Ending request: urn:request:ceea71fb-4089-4dcb-a328-647733b76fe8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:ceea71fb-4089-4dcb-a328-647733b76fe8', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:19:35,397 [270] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:19:35,398 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:19:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:19:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.021 118 0.020)
notificationworker stdout | 2024-06-14 05:19:36,592 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:19:36,592 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:41.591084+00:00 (in 4.998284 seconds)
notificationworker stdout | 2024-06-14 05:19:36,593 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:24:36 UTC)" (scheduled at 2024-06-14 05:19:36.592214+00:00)
notificationworker stdout | 2024-06-14 05:19:36,593 [82] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2024, 6, 14, 5, 19, 36, 593233), 'notification/%'])
notificationworker stdout | 2024-06-14 05:19:36,600 [82] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 19, 36, 593233), True, datetime.datetime(2024, 6, 14, 5, 19, 36, 593233), 0, 'notification/%'])
notificationworker stdout | 2024-06-14 05:19:36,601 [82] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 19, 36, 593233), True, datetime.datetime(2024, 6, 14, 5, 19, 36, 593233), 0, 'notification/%', False, datetime.datetime(2024, 6, 14, 5, 19, 36, 593233), 'notification/%'])
notificationworker stdout | 2024-06-14 05:19:36,602 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:19:36,602 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:24:36 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:19:38,567 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:19:39,326 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:19:41,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:19:41,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:51.591084+00:00 (in 9.999373 seconds)
notificationworker stdout | 2024-06-14 05:19:41,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:51 UTC)" (scheduled at 2024-06-14 05:19:41.591084+00:00)
notificationworker stdout | 2024-06-14 05:19:41,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:19:41,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 41, 592046), True, datetime.datetime(2024, 6, 14, 5, 19, 41, 592046), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:19:41,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:19:41,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:19:41,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:19:51 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:19:42,090 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:19:42,090 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:34.996948+00:00 (in 52.906630 seconds)
repositorygcworker stdout | 2024-06-14 05:19:42,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:42 UTC)" (scheduled at 2024-06-14 05:19:42.089819+00:00)
repositorygcworker stdout | 2024-06-14 05:19:42,090 [109] [DEBUG] [workers.queueworker] Running watchdog.
repositorygcworker stdout | 2024-06-14 05:19:42,090 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:20:42 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:19:44,062 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:19:45,295 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
repositorygcworker stdout | 2024-06-14 05:19:45,897 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:19:46,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:19:46,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:19:58.194620+00:00 (in 12.002513 seconds)
securityworker stdout | 2024-06-14 05:19:46,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:16 UTC)" (scheduled at 2024-06-14 05:19:46.191483+00:00)
securityworker stdout | 2024-06-14 05:19:46,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:19:46,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:19:46,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:19:46,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:19:46,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:19:46,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:19:46,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:16 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:19:46,531 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:19:47,057 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:19:48,407 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:19:48,624 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:19:49,039 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:19:49,044 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:19:49,047 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:19:49,050 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:19:49,052 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:19:50,379 [269] [DEBUG] [app] Starting request: urn:request:3cef2290-4a1c-43bf-96c4-33e48b71c694 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:19:50,379 [271] [DEBUG] [app] Starting request: urn:request:4aa8c485-7688-47db-a39c-a015dac53cbf (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:19:50,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:50,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:19:50,382 [287] [DEBUG] [app] Starting request: urn:request:9710c136-dc27-4cec-b408-0f992a169e72 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:19:50,382 [283] [DEBUG] [app] Starting request: urn:request:08f97f8d-dc3f-433e-affa-58f0953b0b6c (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:19:50,383 [283] [DEBUG] [app] Ending request: urn:request:08f97f8d-dc3f-433e-affa-58f0953b0b6c (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:08f97f8d-dc3f-433e-affa-58f0953b0b6c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:19:50,383 [287] [DEBUG] [app] Ending request: urn:request:9710c136-dc27-4cec-b408-0f992a169e72 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9710c136-dc27-4cec-b408-0f992a169e72', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:19:50,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:19:50,383 [283] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:19:50,383 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:50,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:50,384 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:50,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:19:50,386 [271] [DEBUG] [app] Starting request: urn:request:bdde7da3-e26a-445b-8618-41f643105050 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:50,386 [271] [DEBUG] [app] Ending request: urn:request:bdde7da3-e26a-445b-8618-41f643105050 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bdde7da3-e26a-445b-8618-41f643105050', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:19:50,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:19:50,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:50,387 [269] [DEBUG] [app] Starting request: urn:request:d37a5b76-d8aa-4da7-8c0a-7394d87b2c5d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:19:50,387 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:19:50,387 [269] [DEBUG] [app] Ending request: urn:request:d37a5b76-d8aa-4da7-8c0a-7394d87b2c5d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d37a5b76-d8aa-4da7-8c0a-7394d87b2c5d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:19:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:19:50,387 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:19:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:19:50,387 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:19:50,388 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:19:50,391 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:19:50,392 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:19:50,395 [271] [DEBUG] [app] Ending request: urn:request:4aa8c485-7688-47db-a39c-a015dac53cbf (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4aa8c485-7688-47db-a39c-a015dac53cbf', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:19:50,395 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:19:50,395 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:19:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:19:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.017 118 0.017)
gunicorn-web stdout | 2024-06-14 05:19:50,396 [269] [DEBUG] [app] Ending request: urn:request:3cef2290-4a1c-43bf-96c4-33e48b71c694 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3cef2290-4a1c-43bf-96c4-33e48b71c694', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:19:50,397 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:19:50,397 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:19:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:19:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
notificationworker stdout | 2024-06-14 05:19:51,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:19:51,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:01.591084+00:00 (in 9.999426 seconds)
notificationworker stdout | 2024-06-14 05:19:51,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:01 UTC)" (scheduled at 2024-06-14 05:19:51.591084+00:00)
notificationworker stdout | 2024-06-14 05:19:51,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:19:51,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 19, 51, 592108), True, datetime.datetime(2024, 6, 14, 5, 19, 51, 592108), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:19:51,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:19:51,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:19:51,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:01 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:19:51,953 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:19:52,350 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:19:52,711 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:19:52,822 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
quotaregistrysizeworker stdout | 2024-06-14 05:19:54,960 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:19:55,230 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:19:55,615 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:19:55,795 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
autopruneworker stdout | 2024-06-14 05:19:55,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:19:55,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:25.804090+00:00 (in 29.999406 seconds)
autopruneworker stdout | 2024-06-14 05:19:55,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:25 UTC)" (scheduled at 2024-06-14 05:19:55.804090+00:00)
autopruneworker stdout | 2024-06-14 05:19:55,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342335809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:19:55,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:19:55,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:19:55,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:25 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:19:57,010 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:19:57,671 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:19:57,732 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:19:58,056 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:19:58,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:19:58,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:16.191483+00:00 (in 17.996289 seconds)
securityworker stdout | 2024-06-14 05:19:58,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:28 UTC)" (scheduled at 2024-06-14 05:19:58.194620+00:00)
securityworker stdout | 2024-06-14 05:19:58,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:19:58,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:19:58,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:19:58,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:19:58,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:19:58,198 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:19:58,198 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:19:58,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:19:58,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:19:58,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:19:58,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:19:58,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:19:58,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 14, 58, 204424), 1, 2])
securityworker stdout | 2024-06-14 05:19:58,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:19:58,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:19:58,212 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 14, 58, 204424), 1, 2])
securityworker stdout | 2024-06-14 05:19:58,213 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:19:58,213 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:19:58,213 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:19:58,213 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:19:58,213 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:19:58,214 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:19:58,214 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:19:58,214 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:19:58,214 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:19:58,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:28 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:19:59,015 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:19:59,374 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:19:59,377 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:19:59,380 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:20:00,251 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:20:00,254 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:20:00,256 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:20:00,259 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:20:00,261 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:20:00,266 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:20:00,269 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:20:00,272 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:20:00,274 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
notificationworker stdout | 2024-06-14 05:20:01,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:20:01,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:11.591084+00:00 (in 9.999400 seconds)
notificationworker stdout | 2024-06-14 05:20:01,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:11 UTC)" (scheduled at 2024-06-14 05:20:01.591084+00:00)
notificationworker stdout | 2024-06-14 05:20:01,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:20:01,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 1, 592017), True, datetime.datetime(2024, 6, 14, 5, 20, 1, 592017), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:20:01,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:20:01,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:20:01,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:11 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:20:01,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:20:01,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:31.891735+00:00 (in 29.999381 seconds)
buildlogsarchiver stdout | 2024-06-14 05:20:01,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:31 UTC)" (scheduled at 2024-06-14 05:20:01.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:20:01,894 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 20, 1, 892795), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:20:01,900 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:20:01,900 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:20:01,900 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:31 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:20:02,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:20:02,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:32.991899+00:00 (in 29.999373 seconds)
gcworker stdout | 2024-06-14 05:20:02,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:32 UTC)" (scheduled at 2024-06-14 05:20:02.991899+00:00)
gcworker stdout | 2024-06-14 05:20:02,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132802993, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:20:03,001 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:20:03,001 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:20:03,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:32 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:20:03,799 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:20:03,799 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:08.800520+00:00 (in 5.001100 seconds)
namespacegcworker stdout | 2024-06-14 05:20:03,799 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:03 UTC)" (scheduled at 2024-06-14 05:20:03.798679+00:00)
namespacegcworker stdout | 2024-06-14 05:20:03,799 [75] [DEBUG] [workers.queueworker] Getting work item from queue.
namespacegcworker stdout | 2024-06-14 05:20:03,800 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 3, 799891), True, datetime.datetime(2024, 6, 14, 5, 20, 3, 799891), 0, 'namespacegc/%', 50, 1, 0])
namespacegcworker stdout | 2024-06-14 05:20:03,807 [75] [DEBUG] [workers.queueworker] No more work.
namespacegcworker stdout | 2024-06-14 05:20:03,807 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:20:03,807 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:03 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:20:05,378 [268] [DEBUG] [app] Starting request: urn:request:4d17ddbc-310a-42bd-9b7a-1f48a180c17f (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:20:05,378 [271] [DEBUG] [app] Starting request: urn:request:2681fa2d-53fa-4955-9aa9-bbf4c2d4d58f (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:20:05,379 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:05,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:20:05,381 [280] [DEBUG] [app] Starting request: urn:request:eb3a5f67-7d40-4712-a411-c375cefc4ef6 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:20:05,382 [280] [DEBUG] [app] Ending request: urn:request:eb3a5f67-7d40-4712-a411-c375cefc4ef6 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:eb3a5f67-7d40-4712-a411-c375cefc4ef6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:20:05,382 [287] [DEBUG] [app] Starting request: urn:request:d765201d-60fe-4754-880a-85b37337bd62 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:20:05,382 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:20:05,382 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:20:05,383 [287] [DEBUG] [app] Ending request: urn:request:d765201d-60fe-4754-880a-85b37337bd62 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d765201d-60fe-4754-880a-85b37337bd62', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:20:05,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:20:05,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:05,384 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:05,385 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:05,386 [270] [DEBUG] [app] Starting request: urn:request:2b03304d-c044-4e59-b7f5-fae8b2b6426a (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:20:05,386 [270] [DEBUG] [app] Ending request: urn:request:2b03304d-c044-4e59-b7f5-fae8b2b6426a (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2b03304d-c044-4e59-b7f5-fae8b2b6426a', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:20:05,386 [269] [DEBUG] [app] Starting request: urn:request:2ac5ba99-c424-481c-bc89-33492086e50d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:20:05,387 [269] [DEBUG] [app] Ending request: urn:request:2ac5ba99-c424-481c-bc89-33492086e50d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2ac5ba99-c424-481c-bc89-33492086e50d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:20:05,387 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:20:05,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:05,387 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:20:05,387 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:05,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:20:05,388 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:20:05,391 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:20:05,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:20:05,396 [268] [DEBUG] [app] Ending request: urn:request:4d17ddbc-310a-42bd-9b7a-1f48a180c17f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4d17ddbc-310a-42bd-9b7a-1f48a180c17f', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:20:05,396 [271] [DEBUG] [app] Ending request: urn:request:2681fa2d-53fa-4955-9aa9-bbf4c2d4d58f (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2681fa2d-53fa-4955-9aa9-bbf4c2d4d58f', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:20:05,396 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:20:05,396 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:20:05,397 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:20:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:20:05,397 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:20:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:20:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:20:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
exportactionlogsworker stdout | 2024-06-14 05:20:06,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:20:06,198 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:23.197203+00:00 (in 16.998317 seconds)
exportactionlogsworker stdout | 2024-06-14 05:20:06,199 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:06 UTC)" (scheduled at 2024-06-14 05:20:06.198336+00:00)
exportactionlogsworker stdout | 2024-06-14 05:20:06,199 [66] [DEBUG] [workers.queueworker] Running watchdog.
exportactionlogsworker stdout | 2024-06-14 05:20:06,199 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:06 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:20:07,198 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:20:07,199 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:07.198655+00:00 (in 59.999495 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:20:07,199 [87] [INFO] [apscheduler.executors.default] Running job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:07 UTC)" (scheduled at 2024-06-14 05:20:07.198655+00:00)
quotaregistrysizeworker stdout | 2024-06-14 05:20:07,199 [87] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."size_bytes", "t1"."running", "t1"."queued", "t1"."completed_ms" FROM "quotaregistrysize" AS "t1" LIMIT %s OFFSET %s', [1, 0])
quotaregistrysizeworker stdout | 2024-06-14 05:20:07,204 [87] [DEBUG] [data.database] Disconnecting from database.
quotaregistrysizeworker stdout | 2024-06-14 05:20:07,204 [87] [INFO] [apscheduler.executors.default] Job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:07 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:20:08,603 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
namespacegcworker stdout | 2024-06-14 05:20:08,800 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:20:08,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:15.800790+00:00 (in 6.999595 seconds)
namespacegcworker stdout | 2024-06-14 05:20:08,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:25:08 UTC)" (scheduled at 2024-06-14 05:20:08.800520+00:00)
namespacegcworker stdout | 2024-06-14 05:20:08,802 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2024, 6, 14, 5, 20, 8, 801629), 'namespacegc/%'])
namespacegcworker stdout | 2024-06-14 05:20:08,807 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 20, 8, 801629), True, datetime.datetime(2024, 6, 14, 5, 20, 8, 801629), 0, 'namespacegc/%'])
namespacegcworker stdout | 2024-06-14 05:20:08,809 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 20, 8, 801629), True, datetime.datetime(2024, 6, 14, 5, 20, 8, 801629), 0, 'namespacegc/%', False, datetime.datetime(2024, 6, 14, 5, 20, 8, 801629), 'namespacegc/%'])
namespacegcworker stdout | 2024-06-14 05:20:08,809 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:20:08,810 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:25:08 UTC)" executed successfully
queuecleanupworker stdout | 2024-06-14 05:20:09,348 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:20:11,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:20:11,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:21.591084+00:00 (in 9.999404 seconds)
notificationworker stdout | 2024-06-14 05:20:11,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:21 UTC)" (scheduled at 2024-06-14 05:20:11.591084+00:00)
notificationworker stdout | 2024-06-14 05:20:11,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:20:11,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 11, 592180), True, datetime.datetime(2024, 6, 14, 5, 20, 11, 592180), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:20:11,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:20:11,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:20:11,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:21 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:20:14,095 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:20:15,329 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
namespacegcworker stdout | 2024-06-14 05:20:15,801 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:20:15,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:03.798679+00:00 (in 47.997193 seconds)
namespacegcworker stdout | 2024-06-14 05:20:15,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:15 UTC)" (scheduled at 2024-06-14 05:20:15.800790+00:00)
namespacegcworker stdout | 2024-06-14 05:20:15,801 [75] [DEBUG] [workers.queueworker] Running watchdog.
namespacegcworker stdout | 2024-06-14 05:20:15,801 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:15 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:20:15,913 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:20:16,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:20:16,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:28.194620+00:00 (in 12.002513 seconds)
securityworker stdout | 2024-06-14 05:20:16,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:46 UTC)" (scheduled at 2024-06-14 05:20:16.191483+00:00)
securityworker stdout | 2024-06-14 05:20:16,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:20:16,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:20:16,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:20:16,196 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:20:16,201 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:20:16,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:20:16,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:46 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:20:16,563 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:20:17,094 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:20:18,416 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:20:18,640 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:20:19,048 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:20:19,052 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:20:19,054 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:20:19,057 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:20:19,060 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:20:20,379 [268] [DEBUG] [app] Starting request: urn:request:7ffaa342-bcc7-4f7f-b1b9-1b36e17b3778 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:20:20,379 [269] [DEBUG] [app] Starting request: urn:request:8051bbf1-2002-45ea-afd3-271ef58855b2 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:20:20,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:20,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:20:20,382 [280] [DEBUG] [app] Starting request: urn:request:8af87d38-f470-447a-9d9f-7b5da6648563 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:20:20,382 [287] [DEBUG] [app] Starting request: urn:request:e5886bcc-5f3a-44fe-8566-b84b32ff7795 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:20:20,383 [287] [DEBUG] [app] Ending request: urn:request:e5886bcc-5f3a-44fe-8566-b84b32ff7795 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:e5886bcc-5f3a-44fe-8566-b84b32ff7795', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:20:20,383 [280] [DEBUG] [app] Ending request: urn:request:8af87d38-f470-447a-9d9f-7b5da6648563 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8af87d38-f470-447a-9d9f-7b5da6648563', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:20:20,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:20:20,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:20:20,383 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:20,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:20,384 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:20,384 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:20,386 [270] [DEBUG] [app] Starting request: urn:request:24511733-fc4a-4f44-94e1-6f19999a1970 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:20:20,386 [271] [DEBUG] [app] Starting request: urn:request:ed4f4b08-332c-449a-be50-8cf0fedead06 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:20:20,386 [270] [DEBUG] [app] Ending request: urn:request:24511733-fc4a-4f44-94e1-6f19999a1970 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:24511733-fc4a-4f44-94e1-6f19999a1970', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:20:20,386 [271] [DEBUG] [app] Ending request: urn:request:ed4f4b08-332c-449a-be50-8cf0fedead06 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:ed4f4b08-332c-449a-be50-8cf0fedead06', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:20:20,386 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:20:20,387 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:20,387 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:20:20,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:20,387 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:20:20,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:20:20,390 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:20:20,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:20:20,395 [269] [DEBUG] [app] Ending request: urn:request:8051bbf1-2002-45ea-afd3-271ef58855b2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8051bbf1-2002-45ea-afd3-271ef58855b2', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:20:20,395 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:20:20,395 [268] [DEBUG] [app] Ending request: urn:request:7ffaa342-bcc7-4f7f-b1b9-1b36e17b3778 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:7ffaa342-bcc7-4f7f-b1b9-1b36e17b3778', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:20:20,395 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:20:20,395 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:20:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:20:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.017 118 0.017)
gunicorn-web stdout | 2024-06-14 05:20:20,395 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:20:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:20:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.017 118 0.017)
notificationworker stdout | 2024-06-14 05:20:21,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:20:21,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:21.592513+00:00 (in 0.000907 seconds)
notificationworker stdout | 2024-06-14 05:20:21,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:31 UTC)" (scheduled at 2024-06-14 05:20:21.591084+00:00)
notificationworker stdout | 2024-06-14 05:20:21,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:20:21,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 21, 591906), True, datetime.datetime(2024, 6, 14, 5, 20, 21, 591906), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:20:21,592 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:20:21,593 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:31.591084+00:00 (in 9.998022 seconds)
notificationworker stdout | 2024-06-14 05:20:21,593 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:21 UTC)" (scheduled at 2024-06-14 05:20:21.592513+00:00)
notificationworker stdout | 2024-06-14 05:20:21,593 [82] [DEBUG] [workers.queueworker] Running watchdog.
notificationworker stdout | 2024-06-14 05:20:21,593 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:21 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:20:21,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:20:21,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:20:21,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:31 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:20:21,984 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:20:22,387 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:20:22,747 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:20:22,852 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
exportactionlogsworker stdout | 2024-06-14 05:20:23,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:20:23,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:06.198336+00:00 (in 43.000497 seconds)
exportactionlogsworker stdout | 2024-06-14 05:20:23,198 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:23 UTC)" (scheduled at 2024-06-14 05:20:23.197203+00:00)
exportactionlogsworker stdout | 2024-06-14 05:20:23,198 [66] [DEBUG] [workers.queueworker] Getting work item from queue.
exportactionlogsworker stdout | 2024-06-14 05:20:23,199 [66] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 23, 198254), True, datetime.datetime(2024, 6, 14, 5, 20, 23, 198254), 0, 'exportactionlogs/%', 50, 1, 0])
exportactionlogsworker stdout | 2024-06-14 05:20:23,205 [66] [DEBUG] [workers.queueworker] No more work.
exportactionlogsworker stdout | 2024-06-14 05:20:23,205 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:20:23,205 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:23 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:20:24,997 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:20:25,245 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:20:25,638 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
autopruneworker stdout | 2024-06-14 05:20:25,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:20:25,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:55.804090+00:00 (in 29.999237 seconds)
autopruneworker stdout | 2024-06-14 05:20:25,805 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:55 UTC)" (scheduled at 2024-06-14 05:20:25.804090+00:00)
autopruneworker stdout | 2024-06-14 05:20:25,810 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342365809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:20:25,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:20:25,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:20:25,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:55 UTC)" executed successfully
globalpromstats stdout | 2024-06-14 05:20:25,831 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:20:27,047 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:20:27,709 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:20:27,752 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:20:28,083 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:20:28,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:20:28,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:46.191483+00:00 (in 17.996208 seconds)
securityworker stdout | 2024-06-14 05:20:28,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:58 UTC)" (scheduled at 2024-06-14 05:20:28.194620+00:00)
securityworker stdout | 2024-06-14 05:20:28,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:20:28,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:20:28,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:20:28,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:20:28,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:20:28,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:20:28,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:20:28,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:20:28,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:20:28,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:20:28,210 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:20:28,210 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:20:28,210 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 15, 28, 203757), 1, 2])
securityworker stdout | 2024-06-14 05:20:28,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:20:28,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:20:28,212 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 15, 28, 203757), 1, 2])
securityworker stdout | 2024-06-14 05:20:28,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:20:28,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:20:28,212 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:20:28,212 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:20:28,213 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:20:28,213 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:20:28,213 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:20:28,213 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:20:28,213 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:20:28,213 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:20:58 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:20:29,051 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:20:29,382 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:20:29,385 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:20:29,389 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:20:30,261 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:20:30,264 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:20:30,266 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:20:30,268 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:20:30,271 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:20:30,275 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:20:30,280 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:20:30,283 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:20:30,286 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
securityscanningnotificationworker stdout | 2024-06-14 05:20:31,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:20:31,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:35.332183+00:00 (in 4.000334 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:20:31,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:31 UTC)" (scheduled at 2024-06-14 05:20:31.331172+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:20:31,332 [110] [DEBUG] [workers.queueworker] Getting work item from queue.
securityscanningnotificationworker stdout | 2024-06-14 05:20:31,333 [110] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 31, 332283), True, datetime.datetime(2024, 6, 14, 5, 20, 31, 332283), 0, 'secscanv4/%', 50, 1, 0])
securityscanningnotificationworker stdout | 2024-06-14 05:20:31,339 [110] [DEBUG] [workers.queueworker] No more work.
securityscanningnotificationworker stdout | 2024-06-14 05:20:31,339 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:20:31,339 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:31 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:20:31,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:20:31,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:41.591084+00:00 (in 9.999459 seconds)
notificationworker stdout | 2024-06-14 05:20:31,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:41 UTC)" (scheduled at 2024-06-14 05:20:31.591084+00:00)
notificationworker stdout | 2024-06-14 05:20:31,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:20:31,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 31, 592059), True, datetime.datetime(2024, 6, 14, 5, 20, 31, 592059), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:20:31,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:20:31,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:20:31,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:41 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:20:31,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:20:31,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:01.891735+00:00 (in 29.999210 seconds)
buildlogsarchiver stdout | 2024-06-14 05:20:31,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:01 UTC)" (scheduled at 2024-06-14 05:20:31.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:20:31,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 20, 31, 892935), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:20:31,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:20:31,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:20:31,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:01 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:20:32,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:20:32,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:02.991899+00:00 (in 29.999405 seconds)
gcworker stdout | 2024-06-14 05:20:32,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:02 UTC)" (scheduled at 2024-06-14 05:20:32.991899+00:00)
gcworker stdout | 2024-06-14 05:20:32,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132832992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:20:33,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:20:33,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:20:33,000 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:02 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:20:34,997 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:20:34,997 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:42.089819+00:00 (in 7.092335 seconds)
repositorygcworker stdout | 2024-06-14 05:20:34,997 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:34 UTC)" (scheduled at 2024-06-14 05:20:34.996948+00:00)
repositorygcworker stdout | 2024-06-14 05:20:34,997 [109] [DEBUG] [workers.queueworker] Getting work item from queue.
repositorygcworker stdout | 2024-06-14 05:20:34,998 [109] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 34, 997736), True, datetime.datetime(2024, 6, 14, 5, 20, 34, 997736), 0, 'repositorygc/%', 50, 1, 0])
repositorygcworker stdout | 2024-06-14 05:20:35,005 [109] [DEBUG] [workers.queueworker] No more work.
repositorygcworker stdout | 2024-06-14 05:20:35,005 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:20:35,005 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:34 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:20:35,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:20:35,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:31.331172+00:00 (in 55.998373 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:20:35,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:35 UTC)" (scheduled at 2024-06-14 05:20:35.332183+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:20:35,333 [110] [DEBUG] [workers.queueworker] Running watchdog.
securityscanningnotificationworker stdout | 2024-06-14 05:20:35,333 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:35 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:20:35,378 [268] [DEBUG] [app] Starting request: urn:request:047f6dfd-f51f-42d9-8007-ee217d53dc2b (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:20:35,378 [269] [DEBUG] [app] Starting request: urn:request:4c39760f-9d9d-4e59-ac09-05d1f61cc509 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:20:35,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:35,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:20:35,382 [280] [DEBUG] [app] Starting request: urn:request:d69dd20a-ffc4-44ee-89a8-fa3a7464450b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:20:35,382 [287] [DEBUG] [app] Starting request: urn:request:556d36bd-abc8-4f6c-8508-f184f1d9ac30 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:20:35,382 [280] [DEBUG] [app] Ending request: urn:request:d69dd20a-ffc4-44ee-89a8-fa3a7464450b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:d69dd20a-ffc4-44ee-89a8-fa3a7464450b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:20:35,382 [287] [DEBUG] [app] Ending request: urn:request:556d36bd-abc8-4f6c-8508-f184f1d9ac30 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:556d36bd-abc8-4f6c-8508-f184f1d9ac30', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:20:35,382 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:20:35,382 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.002)
gunicorn-web stdout | 2024-06-14 05:20:35,382 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:35,382 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:35,384 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:35,384 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:35,385 [269] [DEBUG] [app] Starting request: urn:request:7cd3ee15-483f-40f9-a386-49b4c80a02b7 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:20:35,385 [269] [DEBUG] [app] Ending request: urn:request:7cd3ee15-483f-40f9-a386-49b4c80a02b7 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7cd3ee15-483f-40f9-a386-49b4c80a02b7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:20:35,385 [271] [DEBUG] [app] Starting request: urn:request:1921e886-67d3-463b-a1aa-b0f94adeef86 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:20:35,385 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:20:35,386 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:35,386 [271] [DEBUG] [app] Ending request: urn:request:1921e886-67d3-463b-a1aa-b0f94adeef86 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:1921e886-67d3-463b-a1aa-b0f94adeef86', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:20:35,386 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:20:35,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:20:35,386 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:20:35,386 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:20:35,390 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:20:35,390 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:20:35,394 [269] [DEBUG] [app] Ending request: urn:request:4c39760f-9d9d-4e59-ac09-05d1f61cc509 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4c39760f-9d9d-4e59-ac09-05d1f61cc509', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:20:35,394 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:20:35,394 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:20:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:20:35,394 [268] [DEBUG] [app] Ending request: urn:request:047f6dfd-f51f-42d9-8007-ee217d53dc2b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:047f6dfd-f51f-42d9-8007-ee217d53dc2b', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:20:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.017 118 0.017)
gunicorn-web stdout | 2024-06-14 05:20:35,395 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:20:35,395 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:20:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:20:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
chunkcleanupworker stdout | 2024-06-14 05:20:38,639 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:20:39,360 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:20:41,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:20:41,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:51.591084+00:00 (in 9.999408 seconds)
notificationworker stdout | 2024-06-14 05:20:41,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:51 UTC)" (scheduled at 2024-06-14 05:20:41.591084+00:00)
notificationworker stdout | 2024-06-14 05:20:41,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:20:41,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 41, 591985), True, datetime.datetime(2024, 6, 14, 5, 20, 41, 591985), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:20:41,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:20:41,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:20:41,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:20:51 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:20:42,090 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:20:42,090 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:05.089356+00:00 (in 22.999054 seconds)
repositorygcworker stdout | 2024-06-14 05:20:42,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:42 UTC)" (scheduled at 2024-06-14 05:20:42.089819+00:00)
repositorygcworker stdout | 2024-06-14 05:20:42,090 [109] [DEBUG] [workers.queueworker] Running watchdog.
repositorygcworker stdout | 2024-06-14 05:20:42,090 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:21:42 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:20:44,123 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:20:45,353 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
repositorygcworker stdout | 2024-06-14 05:20:45,949 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:20:46,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:20:46,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:20:58.194620+00:00 (in 12.002483 seconds)
securityworker stdout | 2024-06-14 05:20:46,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:16 UTC)" (scheduled at 2024-06-14 05:20:46.191483+00:00)
securityworker stdout | 2024-06-14 05:20:46,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:20:46,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:20:46,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:20:46,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:20:46,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:20:46,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:20:46,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:16 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:20:46,600 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:20:47,112 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:20:48,429 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:20:48,668 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:20:49,056 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:20:49,060 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:20:49,063 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:20:49,066 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:20:49,069 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:20:50,378 [271] [DEBUG] [app] Starting request: urn:request:e579b895-fc82-4709-aa5f-a09b8b29d211 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:20:50,379 [268] [DEBUG] [app] Starting request: urn:request:e0d26ef3-c72e-4094-a9d1-ac75a2f75966 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:20:50,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:50,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:20:50,382 [287] [DEBUG] [app] Starting request: urn:request:02f56f40-33dc-4f00-9dff-a666924cbd51 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:20:50,382 [280] [DEBUG] [app] Starting request: urn:request:f7525508-ef35-4f69-be86-0fc8275d52f1 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:20:50,382 [280] [DEBUG] [app] Ending request: urn:request:f7525508-ef35-4f69-be86-0fc8275d52f1 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f7525508-ef35-4f69-be86-0fc8275d52f1', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:20:50,382 [287] [DEBUG] [app] Ending request: urn:request:02f56f40-33dc-4f00-9dff-a666924cbd51 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:02f56f40-33dc-4f00-9dff-a666924cbd51', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:20:50,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:20:50,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:20:50,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:50,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:20:50,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:50,384 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:20:50,386 [269] [DEBUG] [app] Starting request: urn:request:bb73a1b3-4919-4514-a27b-4f1798c12679 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:20:50,386 [270] [DEBUG] [app] Starting request: urn:request:e1edf2cc-7cb3-41af-9f9a-e9d0059c8343 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:20:50,386 [269] [DEBUG] [app] Ending request: urn:request:bb73a1b3-4919-4514-a27b-4f1798c12679 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bb73a1b3-4919-4514-a27b-4f1798c12679', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:20:50,386 [270] [DEBUG] [app] Ending request: urn:request:e1edf2cc-7cb3-41af-9f9a-e9d0059c8343 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e1edf2cc-7cb3-41af-9f9a-e9d0059c8343', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:20:50,386 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:20:50,386 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:20:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:20:50,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:20:50,386 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:20:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:20:50,387 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:20:50,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:20:50,391 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:20:50,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:20:50,395 [271] [DEBUG] [app] Ending request: urn:request:e579b895-fc82-4709-aa5f-a09b8b29d211 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e579b895-fc82-4709-aa5f-a09b8b29d211', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:20:50,395 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:20:50,396 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:20:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:20:50,396 [268] [DEBUG] [app] Ending request: urn:request:e0d26ef3-c72e-4094-a9d1-ac75a2f75966 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e0d26ef3-c72e-4094-a9d1-ac75a2f75966', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:20:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
gunicorn-web stdout | 2024-06-14 05:20:50,396 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:20:50,396 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:20:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:20:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
notificationworker stdout | 2024-06-14 05:20:51,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:20:51,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:01.591084+00:00 (in 9.999455 seconds)
notificationworker stdout | 2024-06-14 05:20:51,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:01 UTC)" (scheduled at 2024-06-14 05:20:51.591084+00:00)
notificationworker stdout | 2024-06-14 05:20:51,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:20:51,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 20, 51, 592056), True, datetime.datetime(2024, 6, 14, 5, 20, 51, 592056), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:20:51,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:20:51,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:20:51,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:01 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:20:52,012 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:20:52,401 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:20:52,784 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:20:52,888 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
quotaregistrysizeworker stdout | 2024-06-14 05:20:55,035 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:20:55,281 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:20:55,658 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
autopruneworker stdout | 2024-06-14 05:20:55,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:20:55,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:25.804090+00:00 (in 29.999329 seconds)
autopruneworker stdout | 2024-06-14 05:20:55,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:25 UTC)" (scheduled at 2024-06-14 05:20:55.804090+00:00)
autopruneworker stdout | 2024-06-14 05:20:55,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342395809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:20:55,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:20:55,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:20:55,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:25 UTC)" executed successfully
globalpromstats stdout | 2024-06-14 05:20:55,851 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:20:57,063 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:20:57,735 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:20:57,769 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:20:58,120 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:20:58,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:20:58,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:16.191483+00:00 (in 17.996209 seconds)
securityworker stdout | 2024-06-14 05:20:58,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:28 UTC)" (scheduled at 2024-06-14 05:20:58.194620+00:00)
securityworker stdout | 2024-06-14 05:20:58,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:20:58,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:20:58,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:20:58,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:20:58,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:20:58,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:20:58,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:20:58,203 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:20:58,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:20:58,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:20:58,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:20:58,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:20:58,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 15, 58, 204360), 1, 2])
securityworker stdout | 2024-06-14 05:20:58,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:20:58,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:20:58,213 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 15, 58, 204360), 1, 2])
securityworker stdout | 2024-06-14 05:20:58,213 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:20:58,213 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:20:58,213 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:20:58,213 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:20:58,214 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:20:58,214 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:20:58,214 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:20:58,214 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:20:58,214 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:20:58,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:28 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:20:59,071 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:20:59,391 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:20:59,394 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:20:59,398 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:21:00,272 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:21:00,275 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:21:00,277 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:21:00,280 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:21:00,284 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:21:00,288 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:21:00,291 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:21:00,294 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:21:00,297 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
notificationworker stdout | 2024-06-14 05:21:01,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:21:01,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:11.591084+00:00 (in 9.999425 seconds)
notificationworker stdout | 2024-06-14 05:21:01,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:11 UTC)" (scheduled at 2024-06-14 05:21:01.591084+00:00)
notificationworker stdout | 2024-06-14 05:21:01,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:21:01,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 1, 591989), True, datetime.datetime(2024, 6, 14, 5, 21, 1, 591989), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:21:01,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:21:01,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:21:01,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:11 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:21:01,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:21:01,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:31.891735+00:00 (in 29.999315 seconds)
buildlogsarchiver stdout | 2024-06-14 05:21:01,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:31 UTC)" (scheduled at 2024-06-14 05:21:01.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:21:01,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 21, 1, 892781), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:21:01,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:21:01,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:21:01,900 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:31 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:21:02,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:21:02,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:32.991899+00:00 (in 29.999423 seconds)
gcworker stdout | 2024-06-14 05:21:02,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:32 UTC)" (scheduled at 2024-06-14 05:21:02.991899+00:00)
gcworker stdout | 2024-06-14 05:21:02,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132862992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:21:03,001 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:21:03,001 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:21:03,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:32 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:21:03,798 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:21:03,799 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:15.800790+00:00 (in 12.001489 seconds)
namespacegcworker stdout | 2024-06-14 05:21:03,799 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:03 UTC)" (scheduled at 2024-06-14 05:21:03.798679+00:00)
namespacegcworker stdout | 2024-06-14 05:21:03,799 [75] [DEBUG] [workers.queueworker] Getting work item from queue.
namespacegcworker stdout | 2024-06-14 05:21:03,800 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 3, 799749), True, datetime.datetime(2024, 6, 14, 5, 21, 3, 799749), 0, 'namespacegc/%', 50, 1, 0])
namespacegcworker stdout | 2024-06-14 05:21:03,806 [75] [DEBUG] [workers.queueworker] No more work.
namespacegcworker stdout | 2024-06-14 05:21:03,806 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:21:03,807 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:03 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:21:05,089 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:21:05,089 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:34.996948+00:00 (in 29.907069 seconds)
repositorygcworker stdout | 2024-06-14 05:21:05,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:26:05 UTC)" (scheduled at 2024-06-14 05:21:05.089356+00:00)
repositorygcworker stdout | 2024-06-14 05:21:05,090 [109] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2024, 6, 14, 5, 21, 5, 90113), 'repositorygc/%'])
repositorygcworker stdout | 2024-06-14 05:21:05,097 [109] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 21, 5, 90113), True, datetime.datetime(2024, 6, 14, 5, 21, 5, 90113), 0, 'repositorygc/%'])
repositorygcworker stdout | 2024-06-14 05:21:05,098 [109] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 21, 5, 90113), True, datetime.datetime(2024, 6, 14, 5, 21, 5, 90113), 0, 'repositorygc/%', False, datetime.datetime(2024, 6, 14, 5, 21, 5, 90113), 'repositorygc/%'])
repositorygcworker stdout | 2024-06-14 05:21:05,099 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:21:05,099 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:26:05 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:21:05,378 [271] [DEBUG] [app] Starting request: urn:request:59a0830b-a607-4010-97f7-bf5c722799a7 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:21:05,378 [268] [DEBUG] [app] Starting request: urn:request:1380ac65-10a1-4fd8-bf58-1070a9d760d1 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:21:05,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:05,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:21:05,382 [283] [DEBUG] [app] Starting request: urn:request:23f61db8-5cad-4364-87cc-2a8cc9012fcb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:21:05,382 [280] [DEBUG] [app] Starting request: urn:request:8efd0abb-ef3a-42d5-81f8-c2059cb7352f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:21:05,382 [280] [DEBUG] [app] Ending request: urn:request:8efd0abb-ef3a-42d5-81f8-c2059cb7352f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8efd0abb-ef3a-42d5-81f8-c2059cb7352f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:21:05,382 [283] [DEBUG] [app] Ending request: urn:request:23f61db8-5cad-4364-87cc-2a8cc9012fcb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:23f61db8-5cad-4364-87cc-2a8cc9012fcb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:21:05,383 [283] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:21:05,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:21:05,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:05,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:05,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:05,384 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:05,385 [271] [DEBUG] [app] Starting request: urn:request:24873ac3-6fe6-4983-8eec-fdf5db1576cc (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:21:05,386 [271] [DEBUG] [app] Ending request: urn:request:24873ac3-6fe6-4983-8eec-fdf5db1576cc (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:24873ac3-6fe6-4983-8eec-fdf5db1576cc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:21:05,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:05,386 [270] [DEBUG] [app] Starting request: urn:request:8d620116-a797-48dd-bf3e-acc346566217 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:21:05,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:05,386 [270] [DEBUG] [app] Ending request: urn:request:8d620116-a797-48dd-bf3e-acc346566217 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8d620116-a797-48dd-bf3e-acc346566217', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:21:05,386 [271] [INFO] [data.database] Connection pooling disabled for postgresql
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.001)
gunicorn-web stdout | 2024-06-14 05:21:05,387 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:05,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:05,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:21:05,390 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:21:05,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:21:05,395 [268] [DEBUG] [app] Ending request: urn:request:1380ac65-10a1-4fd8-bf58-1070a9d760d1 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:1380ac65-10a1-4fd8-bf58-1070a9d760d1', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:21:05,395 [271] [DEBUG] [app] Ending request: urn:request:59a0830b-a607-4010-97f7-bf5c722799a7 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:59a0830b-a607-4010-97f7-bf5c722799a7', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:21:05,395 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:21:05,395 [271] [DEBUG] [data.database] Disconnecting from database.
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:21:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.018)
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:21:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:21:05,396 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:21:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:21:05,396 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:21:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
exportactionlogsworker stdout | 2024-06-14 05:21:06,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:21:06,198 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:08.198104+00:00 (in 1.999122 seconds)
exportactionlogsworker stdout | 2024-06-14 05:21:06,199 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:06 UTC)" (scheduled at 2024-06-14 05:21:06.198336+00:00)
exportactionlogsworker stdout | 2024-06-14 05:21:06,199 [66] [DEBUG] [workers.queueworker] Running watchdog.
exportactionlogsworker stdout | 2024-06-14 05:21:06,199 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:06 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:21:07,198 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:21:07,199 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:07.198655+00:00 (in 59.999494 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:21:07,199 [87] [INFO] [apscheduler.executors.default] Running job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:07 UTC)" (scheduled at 2024-06-14 05:21:07.198655+00:00)
quotaregistrysizeworker stdout | 2024-06-14 05:21:07,199 [87] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."size_bytes", "t1"."running", "t1"."queued", "t1"."completed_ms" FROM "quotaregistrysize" AS "t1" LIMIT %s OFFSET %s', [1, 0])
quotaregistrysizeworker stdout | 2024-06-14 05:21:07,204 [87] [DEBUG] [data.database] Disconnecting from database.
quotaregistrysizeworker stdout | 2024-06-14 05:21:07,205 [87] [INFO] [apscheduler.executors.default] Job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:07 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:21:08,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:21:08,198 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:23.197203+00:00 (in 14.998443 seconds)
exportactionlogsworker stdout | 2024-06-14 05:21:08,198 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:26:08 UTC)" (scheduled at 2024-06-14 05:21:08.198104+00:00)
exportactionlogsworker stdout | 2024-06-14 05:21:08,199 [66] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2024, 6, 14, 5, 21, 8, 199119), 'exportactionlogs/%'])
exportactionlogsworker stdout | 2024-06-14 05:21:08,205 [66] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 21, 8, 199119), True, datetime.datetime(2024, 6, 14, 5, 21, 8, 199119), 0, 'exportactionlogs/%'])
exportactionlogsworker stdout | 2024-06-14 05:21:08,207 [66] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 21, 8, 199119), True, datetime.datetime(2024, 6, 14, 5, 21, 8, 199119), 0, 'exportactionlogs/%', False, datetime.datetime(2024, 6, 14, 5, 21, 8, 199119), 'exportactionlogs/%'])
exportactionlogsworker stdout | 2024-06-14 05:21:08,208 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:21:08,208 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:26:08 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:21:08,655 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:21:09,396 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:21:11,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:21:11,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:21.591084+00:00 (in 9.999412 seconds)
notificationworker stdout | 2024-06-14 05:21:11,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:21 UTC)" (scheduled at 2024-06-14 05:21:11.591084+00:00)
notificationworker stdout | 2024-06-14 05:21:11,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:21:11,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 11, 592149), True, datetime.datetime(2024, 6, 14, 5, 21, 11, 592149), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:21:11,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:21:11,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:21:11,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:21 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:21:14,160 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:21:15,390 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
namespacegcworker stdout | 2024-06-14 05:21:15,801 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:21:15,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:03.798679+00:00 (in 47.997165 seconds)
namespacegcworker stdout | 2024-06-14 05:21:15,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:15 UTC)" (scheduled at 2024-06-14 05:21:15.800790+00:00)
namespacegcworker stdout | 2024-06-14 05:21:15,801 [75] [DEBUG] [workers.queueworker] Running watchdog.
namespacegcworker stdout | 2024-06-14 05:21:15,801 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:15 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:21:15,974 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:21:16,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:21:16,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:28.194620+00:00 (in 12.002471 seconds)
securityworker stdout | 2024-06-14 05:21:16,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:46 UTC)" (scheduled at 2024-06-14 05:21:16.191483+00:00)
securityworker stdout | 2024-06-14 05:21:16,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:21:16,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:21:16,196 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:21:16,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:21:16,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:21:16,203 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:21:16,203 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:46 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:21:16,638 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:21:17,149 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:21:18,468 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:21:18,693 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:21:19,064 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:21:19,069 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:21:19,072 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:21:19,076 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:21:19,078 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:21:20,378 [270] [DEBUG] [app] Starting request: urn:request:c0d44248-17aa-465a-a990-e771a6143677 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:21:20,379 [269] [DEBUG] [app] Starting request: urn:request:e016c0ab-468e-4b07-b157-0bcc1ac11aff (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:21:20,380 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:20,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:21:20,382 [280] [DEBUG] [app] Starting request: urn:request:ee7dc80f-91f0-4c89-a79b-832611a66e41 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:21:20,382 [287] [DEBUG] [app] Starting request: urn:request:55d641af-8e0d-4832-81f9-0abcebe67e53 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:21:20,382 [280] [DEBUG] [app] Ending request: urn:request:ee7dc80f-91f0-4c89-a79b-832611a66e41 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ee7dc80f-91f0-4c89-a79b-832611a66e41', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:21:20,383 [287] [DEBUG] [app] Ending request: urn:request:55d641af-8e0d-4832-81f9-0abcebe67e53 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:55d641af-8e0d-4832-81f9-0abcebe67e53', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:21:20,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-registry stdout | 2024-06-14 05:21:20,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:20,383 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:20,383 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:21:20,384 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:20,384 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:20,386 [269] [DEBUG] [app] Starting request: urn:request:5ee45e9b-254b-4751-aba8-a68f39833184 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:21:20,386 [268] [DEBUG] [app] Starting request: urn:request:d04fdb80-9006-4812-9988-ada9cc591487 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:21:20,386 [269] [DEBUG] [app] Ending request: urn:request:5ee45e9b-254b-4751-aba8-a68f39833184 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:5ee45e9b-254b-4751-aba8-a68f39833184', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:21:20,386 [268] [DEBUG] [app] Ending request: urn:request:d04fdb80-9006-4812-9988-ada9cc591487 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d04fdb80-9006-4812-9988-ada9cc591487', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:21:20,386 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:20,386 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:21:20,386 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:20,387 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:20,387 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:21:20,387 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:21:20,391 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:21:20,391 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:21:20,395 [269] [DEBUG] [app] Ending request: urn:request:e016c0ab-468e-4b07-b157-0bcc1ac11aff (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:e016c0ab-468e-4b07-b157-0bcc1ac11aff', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:21:20,395 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:21:20,395 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:21:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:21:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:21:20,396 [270] [DEBUG] [app] Ending request: urn:request:c0d44248-17aa-465a-a990-e771a6143677 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:c0d44248-17aa-465a-a990-e771a6143677', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:21:20,397 [270] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:21:20,397 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:21:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:21:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
notificationworker stdout | 2024-06-14 05:21:21,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:21:21,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:21.592513+00:00 (in 0.000885 seconds)
notificationworker stdout | 2024-06-14 05:21:21,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:31 UTC)" (scheduled at 2024-06-14 05:21:21.591084+00:00)
notificationworker stdout | 2024-06-14 05:21:21,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:21:21,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 21, 592003), True, datetime.datetime(2024, 6, 14, 5, 21, 21, 592003), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:21:21,593 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:21:21,593 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:31.591084+00:00 (in 9.997499 seconds)
notificationworker stdout | 2024-06-14 05:21:21,593 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:21 UTC)" (scheduled at 2024-06-14 05:21:21.592513+00:00)
notificationworker stdout | 2024-06-14 05:21:21,593 [82] [DEBUG] [workers.queueworker] Running watchdog.
notificationworker stdout | 2024-06-14 05:21:21,594 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:21 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:21:21,600 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:21:21,600 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:21:21,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:31 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:21:22,049 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:21:22,409 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:21:22,821 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:21:22,897 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
exportactionlogsworker stdout | 2024-06-14 05:21:23,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:21:23,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:06.198336+00:00 (in 43.000492 seconds)
exportactionlogsworker stdout | 2024-06-14 05:21:23,198 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:23 UTC)" (scheduled at 2024-06-14 05:21:23.197203+00:00)
exportactionlogsworker stdout | 2024-06-14 05:21:23,198 [66] [DEBUG] [workers.queueworker] Getting work item from queue.
exportactionlogsworker stdout | 2024-06-14 05:21:23,199 [66] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 23, 198229), True, datetime.datetime(2024, 6, 14, 5, 21, 23, 198229), 0, 'exportactionlogs/%', 50, 1, 0])
exportactionlogsworker stdout | 2024-06-14 05:21:23,205 [66] [DEBUG] [workers.queueworker] No more work.
exportactionlogsworker stdout | 2024-06-14 05:21:23,205 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:21:23,205 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:23 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:21:25,054 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:21:25,289 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:21:25,696 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
autopruneworker stdout | 2024-06-14 05:21:25,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:21:25,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:55.804090+00:00 (in 29.999312 seconds)
autopruneworker stdout | 2024-06-14 05:21:25,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:55 UTC)" (scheduled at 2024-06-14 05:21:25.804090+00:00)
autopruneworker stdout | 2024-06-14 05:21:25,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342425809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:21:25,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:21:25,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:21:25,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:55 UTC)" executed successfully
globalpromstats stdout | 2024-06-14 05:21:25,860 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:21:27,093 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:21:27,771 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:21:27,805 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:21:28,157 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:21:28,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:21:28,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:46.191483+00:00 (in 17.996227 seconds)
securityworker stdout | 2024-06-14 05:21:28,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:58 UTC)" (scheduled at 2024-06-14 05:21:28.194620+00:00)
securityworker stdout | 2024-06-14 05:21:28,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:21:28,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:21:28,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:21:28,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:21:28,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:21:28,198 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:21:28,198 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:21:28,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:21:28,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:21:28,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:21:28,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:21:28,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:21:28,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 16, 28, 204046), 1, 2])
securityworker stdout | 2024-06-14 05:21:28,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:21:28,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:21:28,213 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 16, 28, 204046), 1, 2])
securityworker stdout | 2024-06-14 05:21:28,214 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:21:28,214 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:21:28,214 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:21:28,214 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:21:28,214 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:21:28,214 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:21:28,214 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:21:28,214 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:21:28,214 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:21:28,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:21:58 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:21:29,107 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:21:29,400 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:21:29,404 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:21:29,407 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:21:30,284 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:21:30,287 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:21:30,289 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:21:30,293 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:21:30,295 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:21:30,298 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:21:30,300 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:21:30,304 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:21:30,307 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
securityscanningnotificationworker stdout | 2024-06-14 05:21:31,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:21:31,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:35.332183+00:00 (in 4.000362 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:21:31,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:31 UTC)" (scheduled at 2024-06-14 05:21:31.331172+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:21:31,332 [110] [DEBUG] [workers.queueworker] Getting work item from queue.
securityscanningnotificationworker stdout | 2024-06-14 05:21:31,333 [110] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 31, 332235), True, datetime.datetime(2024, 6, 14, 5, 21, 31, 332235), 0, 'secscanv4/%', 50, 1, 0])
securityscanningnotificationworker stdout | 2024-06-14 05:21:31,339 [110] [DEBUG] [workers.queueworker] No more work.
securityscanningnotificationworker stdout | 2024-06-14 05:21:31,339 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:21:31,339 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:31 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:21:31,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:21:31,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:41.591084+00:00 (in 9.999417 seconds)
notificationworker stdout | 2024-06-14 05:21:31,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:41 UTC)" (scheduled at 2024-06-14 05:21:31.591084+00:00)
notificationworker stdout | 2024-06-14 05:21:31,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:21:31,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 31, 592162), True, datetime.datetime(2024, 6, 14, 5, 21, 31, 592162), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:21:31,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:21:31,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:21:31,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:41 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:21:31,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:21:31,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:01.891735+00:00 (in 29.999409 seconds)
buildlogsarchiver stdout | 2024-06-14 05:21:31,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:01 UTC)" (scheduled at 2024-06-14 05:21:31.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:21:31,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 21, 31, 892717), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:21:31,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:21:31,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:21:31,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:01 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:21:32,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:21:32,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:02.991899+00:00 (in 29.999474 seconds)
gcworker stdout | 2024-06-14 05:21:32,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:02 UTC)" (scheduled at 2024-06-14 05:21:32.991899+00:00)
gcworker stdout | 2024-06-14 05:21:32,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132892992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:21:33,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:21:33,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:21:33,000 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:02 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:21:34,997 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:21:34,997 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:42.089819+00:00 (in 7.092315 seconds)
repositorygcworker stdout | 2024-06-14 05:21:34,997 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:34 UTC)" (scheduled at 2024-06-14 05:21:34.996948+00:00)
repositorygcworker stdout | 2024-06-14 05:21:34,997 [109] [DEBUG] [workers.queueworker] Getting work item from queue.
repositorygcworker stdout | 2024-06-14 05:21:34,998 [109] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 34, 997889), True, datetime.datetime(2024, 6, 14, 5, 21, 34, 997889), 0, 'repositorygc/%', 50, 1, 0])
repositorygcworker stdout | 2024-06-14 05:21:35,004 [109] [DEBUG] [workers.queueworker] No more work.
repositorygcworker stdout | 2024-06-14 05:21:35,004 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:21:35,004 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:34 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:21:35,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:21:35,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:30.331990+00:00 (in 54.999206 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:21:35,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:35 UTC)" (scheduled at 2024-06-14 05:21:35.332183+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:21:35,333 [110] [DEBUG] [workers.queueworker] Running watchdog.
securityscanningnotificationworker stdout | 2024-06-14 05:21:35,333 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:35 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:21:35,378 [268] [DEBUG] [app] Starting request: urn:request:f43eccec-dceb-487a-8e1c-1d22655b3caa (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:21:35,379 [271] [DEBUG] [app] Starting request: urn:request:92060dbd-833f-4800-899e-e69b5fbce384 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:21:35,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:35,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:21:35,382 [280] [DEBUG] [app] Starting request: urn:request:a65f08cd-3a82-4f29-b4ef-15f64d10967f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:21:35,382 [287] [DEBUG] [app] Starting request: urn:request:ca37fdb0-dce6-4a81-9617-2a3e27bbcdd3 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:21:35,383 [280] [DEBUG] [app] Ending request: urn:request:a65f08cd-3a82-4f29-b4ef-15f64d10967f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a65f08cd-3a82-4f29-b4ef-15f64d10967f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:21:35,383 [287] [DEBUG] [app] Ending request: urn:request:ca37fdb0-dce6-4a81-9617-2a3e27bbcdd3 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ca37fdb0-dce6-4a81-9617-2a3e27bbcdd3', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:21:35,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:35,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:21:35,384 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:35,384 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:35,385 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:35,385 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:35,386 [268] [DEBUG] [app] Starting request: urn:request:8b730629-f085-4c7d-95c9-647cd544b885 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:21:35,386 [268] [DEBUG] [app] Ending request: urn:request:8b730629-f085-4c7d-95c9-647cd544b885 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8b730629-f085-4c7d-95c9-647cd544b885', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:21:35,387 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:35,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:35,387 [270] [DEBUG] [app] Starting request: urn:request:bdea3810-7c17-402c-a2da-3e2f088fc608 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:21:35,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:21:35,388 [270] [DEBUG] [app] Ending request: urn:request:bdea3810-7c17-402c-a2da-3e2f088fc608 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bdea3810-7c17-402c-a2da-3e2f088fc608', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:21:35,388 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:35,388 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:35,389 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:21:35,392 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:21:35,393 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:21:35,397 [268] [DEBUG] [app] Ending request: urn:request:f43eccec-dceb-487a-8e1c-1d22655b3caa (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f43eccec-dceb-487a-8e1c-1d22655b3caa', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:21:35,397 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:21:35,397 [271] [DEBUG] [app] Ending request: urn:request:92060dbd-833f-4800-899e-e69b5fbce384 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:92060dbd-833f-4800-899e-e69b5fbce384', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:21:35,397 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:21:35,397 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:21:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:21:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
gunicorn-web stdout | 2024-06-14 05:21:35,398 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:21:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:21:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.021 118 0.021)
chunkcleanupworker stdout | 2024-06-14 05:21:38,687 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:21:39,433 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:21:41,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:21:41,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:51.591084+00:00 (in 9.999433 seconds)
notificationworker stdout | 2024-06-14 05:21:41,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:51 UTC)" (scheduled at 2024-06-14 05:21:41.591084+00:00)
notificationworker stdout | 2024-06-14 05:21:41,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:21:41,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 41, 592134), True, datetime.datetime(2024, 6, 14, 5, 21, 41, 592134), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:21:41,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:21:41,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:21:41,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:21:51 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:21:42,090 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:21:42,090 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:34.996948+00:00 (in 52.906604 seconds)
repositorygcworker stdout | 2024-06-14 05:21:42,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:42 UTC)" (scheduled at 2024-06-14 05:21:42.089819+00:00)
repositorygcworker stdout | 2024-06-14 05:21:42,090 [109] [DEBUG] [workers.queueworker] Running watchdog.
repositorygcworker stdout | 2024-06-14 05:21:42,090 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:22:42 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:21:44,174 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:21:45,427 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
repositorygcworker stdout | 2024-06-14 05:21:45,988 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:21:46,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:21:46,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:21:58.194620+00:00 (in 12.002515 seconds)
securityworker stdout | 2024-06-14 05:21:46,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:16 UTC)" (scheduled at 2024-06-14 05:21:46.191483+00:00)
securityworker stdout | 2024-06-14 05:21:46,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:21:46,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:21:46,196 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:21:46,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:21:46,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:21:46,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:21:46,203 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:16 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:21:46,675 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:21:47,186 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:21:48,487 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:21:48,712 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:21:49,074 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:21:49,078 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:21:49,081 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:21:49,084 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:21:49,087 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:21:50,379 [268] [DEBUG] [app] Starting request: urn:request:138cadcd-c23a-4583-8348-f19e5abde59e (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:21:50,379 [271] [DEBUG] [app] Starting request: urn:request:75c33def-c89b-4693-aa9a-b8b71897550b (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:21:50,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:50,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:21:50,382 [285] [DEBUG] [app] Starting request: urn:request:db25d8ed-f35f-4f89-a8ae-85a3d1c1c68f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:21:50,382 [280] [DEBUG] [app] Starting request: urn:request:55d80a4a-e5c8-4084-baa7-9d2d3f9d8708 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:21:50,383 [285] [DEBUG] [app] Ending request: urn:request:db25d8ed-f35f-4f89-a8ae-85a3d1c1c68f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:db25d8ed-f35f-4f89-a8ae-85a3d1c1c68f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:21:50,383 [280] [DEBUG] [app] Ending request: urn:request:55d80a4a-e5c8-4084-baa7-9d2d3f9d8708 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:55d80a4a-e5c8-4084-baa7-9d2d3f9d8708', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:21:50,383 [285] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:21:50,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:50,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:50,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:21:50,384 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:50,385 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:21:50,386 [270] [DEBUG] [app] Starting request: urn:request:7b19bac6-cbeb-4137-8443-755ea9abdf0b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:21:50,386 [269] [DEBUG] [app] Starting request: urn:request:7bbc5a85-ef1a-4665-8003-cd7c8e202712 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:21:50,386 [270] [DEBUG] [app] Ending request: urn:request:7b19bac6-cbeb-4137-8443-755ea9abdf0b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7b19bac6-cbeb-4137-8443-755ea9abdf0b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:21:50,386 [269] [DEBUG] [app] Ending request: urn:request:7bbc5a85-ef1a-4665-8003-cd7c8e202712 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:7bbc5a85-ef1a-4665-8003-cd7c8e202712', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:21:50,387 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:21:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.001)
gunicorn-web stdout | 2024-06-14 05:21:50,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:50,387 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:21:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:21:50,387 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:21:50,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:21:50,387 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:21:50,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:21:50,392 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:21:50,395 [268] [DEBUG] [app] Ending request: urn:request:138cadcd-c23a-4583-8348-f19e5abde59e (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:138cadcd-c23a-4583-8348-f19e5abde59e', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:21:50,395 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:21:50,396 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:21:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:21:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:21:50,396 [271] [DEBUG] [app] Ending request: urn:request:75c33def-c89b-4693-aa9a-b8b71897550b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:75c33def-c89b-4693-aa9a-b8b71897550b', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:21:50,396 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:21:50,397 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:21:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:21:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.020)
notificationworker stdout | 2024-06-14 05:21:51,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:21:51,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:01.591084+00:00 (in 9.999509 seconds)
notificationworker stdout | 2024-06-14 05:21:51,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:01 UTC)" (scheduled at 2024-06-14 05:21:51.591084+00:00)
notificationworker stdout | 2024-06-14 05:21:51,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:21:51,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 21, 51, 591964), True, datetime.datetime(2024, 6, 14, 5, 21, 51, 591964), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:21:51,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:21:51,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:21:51,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:01 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:21:52,086 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:21:52,446 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:21:52,858 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:21:52,920 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
quotaregistrysizeworker stdout | 2024-06-14 05:21:55,080 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:21:55,307 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:21:55,729 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
autopruneworker stdout | 2024-06-14 05:21:55,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:21:55,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:25.804090+00:00 (in 29.999324 seconds)
autopruneworker stdout | 2024-06-14 05:21:55,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:25 UTC)" (scheduled at 2024-06-14 05:21:55.804090+00:00)
autopruneworker stdout | 2024-06-14 05:21:55,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342455808, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:21:55,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:21:55,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:21:55,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:25 UTC)" executed successfully
globalpromstats stdout | 2024-06-14 05:21:55,897 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:21:57,126 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:21:57,807 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:21:57,843 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
autopruneworker stdout | 2024-06-14 05:21:58,193 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityworker stdout | 2024-06-14 05:21:58,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:21:58,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:16.191483+00:00 (in 17.996283 seconds)
securityworker stdout | 2024-06-14 05:21:58,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:28 UTC)" (scheduled at 2024-06-14 05:21:58.194620+00:00)
securityworker stdout | 2024-06-14 05:21:58,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:21:58,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:21:58,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:21:58,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:21:58,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:21:58,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:21:58,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:21:58,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:21:58,203 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:21:58,208 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:21:58,209 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:21:58,209 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:21:58,210 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 16, 58, 203269), 1, 2])
securityworker stdout | 2024-06-14 05:21:58,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:21:58,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:21:58,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 16, 58, 203269), 1, 2])
securityworker stdout | 2024-06-14 05:21:58,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:21:58,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:21:58,212 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:21:58,212 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:21:58,212 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:21:58,212 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:21:58,212 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:21:58,212 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:21:58,212 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:21:58,213 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:28 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:21:59,132 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:21:59,406 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:21:59,412 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:21:59,414 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:22:00,294 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:22:00,297 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:22:00,299 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:22:00,301 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:22:00,304 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:22:00,308 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:22:00,312 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:22:00,316 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:22:00,319 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
notificationworker stdout | 2024-06-14 05:22:01,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:22:01,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:11.591084+00:00 (in 9.999462 seconds)
notificationworker stdout | 2024-06-14 05:22:01,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:11 UTC)" (scheduled at 2024-06-14 05:22:01.591084+00:00)
notificationworker stdout | 2024-06-14 05:22:01,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:22:01,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 1, 592100), True, datetime.datetime(2024, 6, 14, 5, 22, 1, 592100), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:22:01,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:22:01,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:22:01,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:11 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:22:01,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:22:01,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:31.891735+00:00 (in 29.999386 seconds)
buildlogsarchiver stdout | 2024-06-14 05:22:01,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:31 UTC)" (scheduled at 2024-06-14 05:22:01.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:22:01,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 22, 1, 892739), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:22:01,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:22:01,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:22:01,900 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:31 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:22:02,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:22:02,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:32.991899+00:00 (in 29.999447 seconds)
gcworker stdout | 2024-06-14 05:22:02,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:32 UTC)" (scheduled at 2024-06-14 05:22:02.991899+00:00)
gcworker stdout | 2024-06-14 05:22:02,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132922992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:22:03,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:22:03,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:22:03,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:32 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:22:03,798 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:22:03,799 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:15.800790+00:00 (in 12.001527 seconds)
namespacegcworker stdout | 2024-06-14 05:22:03,799 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:03 UTC)" (scheduled at 2024-06-14 05:22:03.798679+00:00)
namespacegcworker stdout | 2024-06-14 05:22:03,799 [75] [DEBUG] [workers.queueworker] Getting work item from queue.
namespacegcworker stdout | 2024-06-14 05:22:03,800 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 3, 799671), True, datetime.datetime(2024, 6, 14, 5, 22, 3, 799671), 0, 'namespacegc/%', 50, 1, 0])
namespacegcworker stdout | 2024-06-14 05:22:03,806 [75] [DEBUG] [workers.queueworker] No more work.
namespacegcworker stdout | 2024-06-14 05:22:03,806 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:22:03,806 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:03 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:22:05,378 [270] [DEBUG] [app] Starting request: urn:request:cbcf44fc-335e-48b5-99a9-7a4e6f659760 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:22:05,378 [271] [DEBUG] [app] Starting request: urn:request:d09d48e5-3553-48ad-9a12-3695ce952049 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:22:05,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:05,380 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:22:05,381 [280] [DEBUG] [app] Starting request: urn:request:2e5c9b9f-803f-4536-bc52-c188a63b2b39 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:22:05,382 [287] [DEBUG] [app] Starting request: urn:request:22dbe1f1-7d6a-409c-b8f9-824053a01d09 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:22:05,382 [280] [DEBUG] [app] Ending request: urn:request:2e5c9b9f-803f-4536-bc52-c188a63b2b39 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:2e5c9b9f-803f-4536-bc52-c188a63b2b39', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:22:05,382 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:22:05,382 [287] [DEBUG] [app] Ending request: urn:request:22dbe1f1-7d6a-409c-b8f9-824053a01d09 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:22dbe1f1-7d6a-409c-b8f9-824053a01d09', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:22:05,382 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:22:05,382 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:22:05,383 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-web stdout | 2024-06-14 05:22:05,383 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:05,384 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:05,385 [271] [DEBUG] [app] Starting request: urn:request:789ab9ac-31c9-4ac0-ba31-13d9fd90cd39 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:22:05,385 [271] [DEBUG] [app] Ending request: urn:request:789ab9ac-31c9-4ac0-ba31-13d9fd90cd39 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:789ab9ac-31c9-4ac0-ba31-13d9fd90cd39', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:22:05,385 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:22:05,385 [268] [DEBUG] [app] Starting request: urn:request:a4977ee5-6483-45ed-a77b-41afb645d569 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:22:05,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:05,386 [268] [DEBUG] [app] Ending request: urn:request:a4977ee5-6483-45ed-a77b-41afb645d569 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:a4977ee5-6483-45ed-a77b-41afb645d569', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:22:05,386 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:22:05,386 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:22:05,386 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:22:05,387 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:22:05,390 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:22:05,390 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:22:05,394 [270] [DEBUG] [app] Ending request: urn:request:cbcf44fc-335e-48b5-99a9-7a4e6f659760 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cbcf44fc-335e-48b5-99a9-7a4e6f659760', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:22:05,394 [271] [DEBUG] [app] Ending request: urn:request:d09d48e5-3553-48ad-9a12-3695ce952049 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d09d48e5-3553-48ad-9a12-3695ce952049', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:22:05,394 [270] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:22:05,395 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:22:05,395 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:22:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:22:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.017)
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:22:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:22:05,395 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:22:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
exportactionlogsworker stdout | 2024-06-14 05:22:06,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:22:06,198 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:23.197203+00:00 (in 16.998317 seconds)
exportactionlogsworker stdout | 2024-06-14 05:22:06,199 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:06 UTC)" (scheduled at 2024-06-14 05:22:06.198336+00:00)
exportactionlogsworker stdout | 2024-06-14 05:22:06,199 [66] [DEBUG] [workers.queueworker] Running watchdog.
exportactionlogsworker stdout | 2024-06-14 05:22:06,199 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:06 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:22:07,198 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:22:07,199 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:07.198655+00:00 (in 59.999469 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:22:07,199 [87] [INFO] [apscheduler.executors.default] Running job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:07 UTC)" (scheduled at 2024-06-14 05:22:07.198655+00:00)
quotaregistrysizeworker stdout | 2024-06-14 05:22:07,200 [87] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."size_bytes", "t1"."running", "t1"."queued", "t1"."completed_ms" FROM "quotaregistrysize" AS "t1" LIMIT %s OFFSET %s', [1, 0])
quotaregistrysizeworker stdout | 2024-06-14 05:22:07,205 [87] [DEBUG] [data.database] Disconnecting from database.
quotaregistrysizeworker stdout | 2024-06-14 05:22:07,205 [87] [INFO] [apscheduler.executors.default] Job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:07 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:22:08,705 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:22:09,440 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:22:11,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:22:11,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:21.591084+00:00 (in 9.999468 seconds)
notificationworker stdout | 2024-06-14 05:22:11,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:21 UTC)" (scheduled at 2024-06-14 05:22:11.591084+00:00)
notificationworker stdout | 2024-06-14 05:22:11,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:22:11,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 11, 592077), True, datetime.datetime(2024, 6, 14, 5, 22, 11, 592077), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:22:11,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:22:11,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:22:11,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:21 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:22:14,184 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:22:15,447 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
namespacegcworker stdout | 2024-06-14 05:22:15,801 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:22:15,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:03.798679+00:00 (in 47.997271 seconds)
namespacegcworker stdout | 2024-06-14 05:22:15,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:15 UTC)" (scheduled at 2024-06-14 05:22:15.800790+00:00)
namespacegcworker stdout | 2024-06-14 05:22:15,801 [75] [DEBUG] [workers.queueworker] Running watchdog.
namespacegcworker stdout | 2024-06-14 05:22:15,801 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:15 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:22:15,999 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:22:16,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:22:16,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:28.194620+00:00 (in 12.002571 seconds)
securityworker stdout | 2024-06-14 05:22:16,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:46 UTC)" (scheduled at 2024-06-14 05:22:16.191483+00:00)
securityworker stdout | 2024-06-14 05:22:16,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:22:16,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:22:16,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:22:16,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:22:16,201 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:22:16,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:22:16,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:46 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:22:16,712 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:22:17,223 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:22:18,498 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:22:18,734 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:22:19,084 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:22:19,088 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:22:19,091 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:22:19,093 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:22:19,097 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:22:20,378 [270] [DEBUG] [app] Starting request: urn:request:10bf11fa-7c3f-4214-95df-1e529ff9457b (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:22:20,378 [269] [DEBUG] [app] Starting request: urn:request:b89bc66e-16bf-4156-82da-d785e2b4e688 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:22:20,380 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:20,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:22:20,382 [280] [DEBUG] [app] Starting request: urn:request:efb72d5f-eb43-40cd-8e48-88f8043497ca (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:22:20,382 [283] [DEBUG] [app] Starting request: urn:request:9396a86f-6c00-4827-bb9a-13b7c6c2e4e5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:22:20,382 [280] [DEBUG] [app] Ending request: urn:request:efb72d5f-eb43-40cd-8e48-88f8043497ca (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:efb72d5f-eb43-40cd-8e48-88f8043497ca', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:22:20,382 [283] [DEBUG] [app] Ending request: urn:request:9396a86f-6c00-4827-bb9a-13b7c6c2e4e5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:9396a86f-6c00-4827-bb9a-13b7c6c2e4e5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:22:20,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.001)
gunicorn-registry stdout | 2024-06-14 05:22:20,383 [283] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:22:20,383 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:20,383 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:22:20,384 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:20,385 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:20,385 [270] [DEBUG] [app] Starting request: urn:request:c7965f36-6a7c-46db-9c45-00680081fc32 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:22:20,386 [270] [DEBUG] [app] Ending request: urn:request:c7965f36-6a7c-46db-9c45-00680081fc32 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:c7965f36-6a7c-46db-9c45-00680081fc32', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:22:20,386 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:22:20,386 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:22:20,386 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:22:20,387 [271] [DEBUG] [app] Starting request: urn:request:e81f0e92-df0c-4590-9aa2-d1c80216e910 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:22:20,387 [271] [DEBUG] [app] Ending request: urn:request:e81f0e92-df0c-4590-9aa2-d1c80216e910 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e81f0e92-df0c-4590-9aa2-d1c80216e910', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:22:20,388 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.001)
gunicorn-web stdout | 2024-06-14 05:22:20,388 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:20,388 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:22:20,391 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:22:20,392 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:22:20,395 [270] [DEBUG] [app] Ending request: urn:request:10bf11fa-7c3f-4214-95df-1e529ff9457b (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:10bf11fa-7c3f-4214-95df-1e529ff9457b', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:22:20,395 [270] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:22:20,396 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:22:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:22:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.018)
gunicorn-web stdout | 2024-06-14 05:22:20,397 [269] [DEBUG] [app] Ending request: urn:request:b89bc66e-16bf-4156-82da-d785e2b4e688 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:b89bc66e-16bf-4156-82da-d785e2b4e688', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:22:20,397 [269] [DEBUG] [data.database] Disconnecting from database.
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:22:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
gunicorn-web stdout | 2024-06-14 05:22:20,398 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:22:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
notificationworker stdout | 2024-06-14 05:22:21,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:22:21,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:21.592513+00:00 (in 0.000944 seconds)
notificationworker stdout | 2024-06-14 05:22:21,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:31 UTC)" (scheduled at 2024-06-14 05:22:21.591084+00:00)
notificationworker stdout | 2024-06-14 05:22:21,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:22:21,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 21, 592052), True, datetime.datetime(2024, 6, 14, 5, 22, 21, 592052), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:22:21,593 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:22:21,593 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:31.591084+00:00 (in 9.997507 seconds)
notificationworker stdout | 2024-06-14 05:22:21,593 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:21 UTC)" (scheduled at 2024-06-14 05:22:21.592513+00:00)
notificationworker stdout | 2024-06-14 05:22:21,593 [82] [DEBUG] [workers.queueworker] Running watchdog.
notificationworker stdout | 2024-06-14 05:22:21,593 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:21 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:22:21,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:22:21,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:22:21,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:31 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:22:22,124 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:22:22,466 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:22:22,867 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:22:22,957 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
exportactionlogsworker stdout | 2024-06-14 05:22:23,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:22:23,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:06.198336+00:00 (in 43.000518 seconds)
exportactionlogsworker stdout | 2024-06-14 05:22:23,198 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:23 UTC)" (scheduled at 2024-06-14 05:22:23.197203+00:00)
exportactionlogsworker stdout | 2024-06-14 05:22:23,198 [66] [DEBUG] [workers.queueworker] Getting work item from queue.
exportactionlogsworker stdout | 2024-06-14 05:22:23,199 [66] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 23, 198222), True, datetime.datetime(2024, 6, 14, 5, 22, 23, 198222), 0, 'exportactionlogs/%', 50, 1, 0])
exportactionlogsworker stdout | 2024-06-14 05:22:23,205 [66] [DEBUG] [workers.queueworker] No more work.
exportactionlogsworker stdout | 2024-06-14 05:22:23,205 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:22:23,205 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:23 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:22:25,094 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:22:25,344 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:22:25,748 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
autopruneworker stdout | 2024-06-14 05:22:25,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:22:25,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:55.804090+00:00 (in 29.999375 seconds)
autopruneworker stdout | 2024-06-14 05:22:25,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:55 UTC)" (scheduled at 2024-06-14 05:22:25.804090+00:00)
autopruneworker stdout | 2024-06-14 05:22:25,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342485808, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:22:25,811 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:22:25,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:22:25,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:55 UTC)" executed successfully
globalpromstats stdout | 2024-06-14 05:22:25,907 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:22:27,163 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:22:27,841 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:22:27,879 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
securityworker stdout | 2024-06-14 05:22:28,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:22:28,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:46.191483+00:00 (in 17.996316 seconds)
securityworker stdout | 2024-06-14 05:22:28,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:58 UTC)" (scheduled at 2024-06-14 05:22:28.194620+00:00)
securityworker stdout | 2024-06-14 05:22:28,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:22:28,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:22:28,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:22:28,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:22:28,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:22:28,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:22:28,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:22:28,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:22:28,203 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:22:28,208 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:22:28,210 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:22:28,210 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:22:28,210 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 17, 28, 203652), 1, 2])
securityworker stdout | 2024-06-14 05:22:28,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:22:28,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:22:28,212 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 17, 28, 203652), 1, 2])
securityworker stdout | 2024-06-14 05:22:28,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:22:28,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:22:28,212 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:22:28,212 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:22:28,212 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:22:28,213 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:22:28,213 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:22:28,213 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:22:28,213 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:22:28,213 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:22:58 UTC)" executed successfully
autopruneworker stdout | 2024-06-14 05:22:28,231 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityscanningnotificationworker stdout | 2024-06-14 05:22:29,161 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:22:29,414 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:22:29,420 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:22:29,423 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:22:30,305 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:22:30,307 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:22:30,311 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:22:30,313 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:22:30,317 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:22:30,322 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:22:30,325 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:22:30,327 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:22:30,330 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
securityscanningnotificationworker stdout | 2024-06-14 05:22:30,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:22:30,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:31.331172+00:00 (in 0.998644 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:22:30,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:27:30 UTC)" (scheduled at 2024-06-14 05:22:30.331990+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:22:30,333 [110] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2024, 6, 14, 5, 22, 30, 332877), 'secscanv4/%'])
securityscanningnotificationworker stdout | 2024-06-14 05:22:30,339 [110] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 22, 30, 332877), True, datetime.datetime(2024, 6, 14, 5, 22, 30, 332877), 0, 'secscanv4/%'])
securityscanningnotificationworker stdout | 2024-06-14 05:22:30,340 [110] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 22, 30, 332877), True, datetime.datetime(2024, 6, 14, 5, 22, 30, 332877), 0, 'secscanv4/%', False, datetime.datetime(2024, 6, 14, 5, 22, 30, 332877), 'secscanv4/%'])
securityscanningnotificationworker stdout | 2024-06-14 05:22:30,341 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:22:30,341 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:27:30 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:22:31,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:22:31,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:35.332183+00:00 (in 4.000371 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:22:31,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:31 UTC)" (scheduled at 2024-06-14 05:22:31.331172+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:22:31,332 [110] [DEBUG] [workers.queueworker] Getting work item from queue.
securityscanningnotificationworker stdout | 2024-06-14 05:22:31,333 [110] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 31, 332223), True, datetime.datetime(2024, 6, 14, 5, 22, 31, 332223), 0, 'secscanv4/%', 50, 1, 0])
securityscanningnotificationworker stdout | 2024-06-14 05:22:31,339 [110] [DEBUG] [workers.queueworker] No more work.
securityscanningnotificationworker stdout | 2024-06-14 05:22:31,339 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:22:31,339 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:31 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:22:31,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:22:31,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:41.591084+00:00 (in 9.999475 seconds)
notificationworker stdout | 2024-06-14 05:22:31,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:41 UTC)" (scheduled at 2024-06-14 05:22:31.591084+00:00)
notificationworker stdout | 2024-06-14 05:22:31,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:22:31,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 31, 592079), True, datetime.datetime(2024, 6, 14, 5, 22, 31, 592079), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:22:31,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:22:31,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:22:31,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:41 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:22:31,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:22:31,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:01.891735+00:00 (in 29.999387 seconds)
buildlogsarchiver stdout | 2024-06-14 05:22:31,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:01 UTC)" (scheduled at 2024-06-14 05:22:31.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:22:31,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 22, 31, 892750), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:22:31,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:22:31,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:22:31,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:01 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:22:32,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:22:32,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:02.991899+00:00 (in 29.999484 seconds)
gcworker stdout | 2024-06-14 05:22:32,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:02 UTC)" (scheduled at 2024-06-14 05:22:32.991899+00:00)
gcworker stdout | 2024-06-14 05:22:32,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132952992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:22:33,001 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:22:33,001 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:22:33,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:02 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:22:34,997 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:22:34,997 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:42.089819+00:00 (in 7.092264 seconds)
repositorygcworker stdout | 2024-06-14 05:22:34,997 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:34 UTC)" (scheduled at 2024-06-14 05:22:34.996948+00:00)
repositorygcworker stdout | 2024-06-14 05:22:34,997 [109] [DEBUG] [workers.queueworker] Getting work item from queue.
repositorygcworker stdout | 2024-06-14 05:22:34,998 [109] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 34, 997997), True, datetime.datetime(2024, 6, 14, 5, 22, 34, 997997), 0, 'repositorygc/%', 50, 1, 0])
repositorygcworker stdout | 2024-06-14 05:22:35,004 [109] [DEBUG] [workers.queueworker] No more work.
repositorygcworker stdout | 2024-06-14 05:22:35,004 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:22:35,004 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:34 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:22:35,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:22:35,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:31.331172+00:00 (in 55.998421 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:22:35,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:35 UTC)" (scheduled at 2024-06-14 05:22:35.332183+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:22:35,333 [110] [DEBUG] [workers.queueworker] Running watchdog.
securityscanningnotificationworker stdout | 2024-06-14 05:22:35,333 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:35 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:22:35,378 [270] [DEBUG] [app] Starting request: urn:request:bc12b7db-07c1-454e-9042-fdddeb89d3fd (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:22:35,378 [269] [DEBUG] [app] Starting request: urn:request:0aebbdfc-8d84-429c-a6b1-a8c756dd4097 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:22:35,380 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:35,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:22:35,381 [286] [DEBUG] [app] Starting request: urn:request:c80d4b3f-7ac0-435c-b33a-156d603c689b (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:22:35,382 [280] [DEBUG] [app] Starting request: urn:request:ac4df766-bfd7-45d8-ab22-1b6bc651b63d (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:22:35,382 [286] [DEBUG] [app] Ending request: urn:request:c80d4b3f-7ac0-435c-b33a-156d603c689b (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:c80d4b3f-7ac0-435c-b33a-156d603c689b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:22:35,382 [286] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:22:35,382 [280] [DEBUG] [app] Ending request: urn:request:ac4df766-bfd7-45d8-ab22-1b6bc651b63d (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ac4df766-bfd7-45d8-ab22-1b6bc651b63d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.002)
gunicorn-web stdout | 2024-06-14 05:22:35,382 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:22:35,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:22:35,383 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:35,384 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:35,384 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:35,385 [270] [DEBUG] [app] Starting request: urn:request:b0f7b06d-fd73-46d2-9e9e-a9d3955df085 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:22:35,385 [270] [DEBUG] [app] Ending request: urn:request:b0f7b06d-fd73-46d2-9e9e-a9d3955df085 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b0f7b06d-fd73-46d2-9e9e-a9d3955df085', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:22:35,386 [268] [DEBUG] [app] Starting request: urn:request:d119e5b5-bf7f-40e4-ad65-e024602d1d9d (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:22:35,386 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:22:35,386 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:35,386 [268] [DEBUG] [app] Ending request: urn:request:d119e5b5-bf7f-40e4-ad65-e024602d1d9d (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:d119e5b5-bf7f-40e4-ad65-e024602d1d9d', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:22:35,386 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:22:35,386 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:22:35,387 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:35,387 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:22:35,390 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:22:35,391 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:22:35,395 [270] [DEBUG] [app] Ending request: urn:request:bc12b7db-07c1-454e-9042-fdddeb89d3fd (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bc12b7db-07c1-454e-9042-fdddeb89d3fd', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:22:35,395 [269] [DEBUG] [app] Ending request: urn:request:0aebbdfc-8d84-429c-a6b1-a8c756dd4097 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0aebbdfc-8d84-429c-a6b1-a8c756dd4097', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:22:35,395 [270] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:22:35,395 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:22:35,395 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:22:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:22:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:22:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:22:35,396 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:22:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
chunkcleanupworker stdout | 2024-06-14 05:22:38,740 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:22:39,455 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:22:41,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:22:41,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:51.591084+00:00 (in 9.999448 seconds)
notificationworker stdout | 2024-06-14 05:22:41,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:51 UTC)" (scheduled at 2024-06-14 05:22:41.591084+00:00)
notificationworker stdout | 2024-06-14 05:22:41,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:22:41,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 41, 592072), True, datetime.datetime(2024, 6, 14, 5, 22, 41, 592072), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:22:41,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:22:41,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:22:41,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:22:51 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:22:42,090 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:22:42,090 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:34.996948+00:00 (in 52.906605 seconds)
repositorygcworker stdout | 2024-06-14 05:22:42,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:42 UTC)" (scheduled at 2024-06-14 05:22:42.089819+00:00)
repositorygcworker stdout | 2024-06-14 05:22:42,090 [109] [DEBUG] [workers.queueworker] Running watchdog.
repositorygcworker stdout | 2024-06-14 05:22:42,090 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:23:42 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:22:44,201 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:22:45,483 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
repositorygcworker stdout | 2024-06-14 05:22:46,036 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:22:46,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:22:46,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:22:58.194620+00:00 (in 12.002531 seconds)
securityworker stdout | 2024-06-14 05:22:46,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:16 UTC)" (scheduled at 2024-06-14 05:22:46.191483+00:00)
securityworker stdout | 2024-06-14 05:22:46,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:22:46,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:22:46,196 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:22:46,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:22:46,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:22:46,203 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:22:46,203 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:16 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:22:46,721 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:22:47,260 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:22:48,530 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:22:48,744 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:22:49,093 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:22:49,098 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:22:49,100 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:22:49,102 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:22:49,106 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:22:50,379 [269] [DEBUG] [app] Starting request: urn:request:2daf9209-6032-4c7f-9690-030c71fb300d (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:22:50,379 [271] [DEBUG] [app] Starting request: urn:request:d9a66633-1cb6-4a1e-8e65-bc4fa896bd25 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:22:50,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:50,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:22:50,382 [287] [DEBUG] [app] Starting request: urn:request:1d32a591-6e12-48df-b0af-0b8694313c12 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:22:50,382 [280] [DEBUG] [app] Starting request: urn:request:53f3655f-7b0d-475b-a945-bb596de0f363 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:22:50,383 [287] [DEBUG] [app] Ending request: urn:request:1d32a591-6e12-48df-b0af-0b8694313c12 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:1d32a591-6e12-48df-b0af-0b8694313c12', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:22:50,383 [280] [DEBUG] [app] Ending request: urn:request:53f3655f-7b0d-475b-a945-bb596de0f363 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:53f3655f-7b0d-475b-a945-bb596de0f363', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:22:50,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:22:50,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:22:50,383 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:50,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:50,384 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:50,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:22:50,386 [271] [DEBUG] [app] Starting request: urn:request:9491f904-b1a8-4b91-acc3-0958dd79172b (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:22:50,386 [271] [DEBUG] [app] Ending request: urn:request:9491f904-b1a8-4b91-acc3-0958dd79172b (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9491f904-b1a8-4b91-acc3-0958dd79172b', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:22:50,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:22:50,386 [268] [DEBUG] [app] Starting request: urn:request:2ad2e129-50f9-47c7-9e7f-a182028db299 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:22:50,386 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:22:50,387 [268] [DEBUG] [app] Ending request: urn:request:2ad2e129-50f9-47c7-9e7f-a182028db299 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:2ad2e129-50f9-47c7-9e7f-a182028db299', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:22:50,387 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:22:50,387 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:22:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:22:50,387 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:22:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:22:50,387 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:22:50,391 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:22:50,391 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:22:50,395 [271] [DEBUG] [app] Ending request: urn:request:d9a66633-1cb6-4a1e-8e65-bc4fa896bd25 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:d9a66633-1cb6-4a1e-8e65-bc4fa896bd25', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:22:50,395 [269] [DEBUG] [app] Ending request: urn:request:2daf9209-6032-4c7f-9690-030c71fb300d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:2daf9209-6032-4c7f-9690-030c71fb300d', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:22:50,396 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:22:50,396 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:22:50,396 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:22:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:22:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:22:50,396 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:22:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:22:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
notificationworker stdout | 2024-06-14 05:22:51,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:22:51,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:01.591084+00:00 (in 9.999468 seconds)
notificationworker stdout | 2024-06-14 05:22:51,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:01 UTC)" (scheduled at 2024-06-14 05:22:51.591084+00:00)
notificationworker stdout | 2024-06-14 05:22:51,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:22:51,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 22, 51, 592112), True, datetime.datetime(2024, 6, 14, 5, 22, 51, 592112), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:22:51,600 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:22:51,600 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:22:51,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:01 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:22:52,159 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:22:52,484 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:22:52,904 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:22:52,994 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
quotaregistrysizeworker stdout | 2024-06-14 05:22:55,118 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:22:55,374 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
notificationworker stdout | 2024-06-14 05:22:55,777 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
autopruneworker stdout | 2024-06-14 05:22:55,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:22:55,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:25.804090+00:00 (in 29.999356 seconds)
autopruneworker stdout | 2024-06-14 05:22:55,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:25 UTC)" (scheduled at 2024-06-14 05:22:55.804090+00:00)
autopruneworker stdout | 2024-06-14 05:22:55,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342515808, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:22:55,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:22:55,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:22:55,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:25 UTC)" executed successfully
globalpromstats stdout | 2024-06-14 05:22:55,920 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:22:57,187 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:22:57,851 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:22:57,912 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
securityworker stdout | 2024-06-14 05:22:58,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:22:58,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:16.191483+00:00 (in 17.996250 seconds)
securityworker stdout | 2024-06-14 05:22:58,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:28 UTC)" (scheduled at 2024-06-14 05:22:58.194620+00:00)
securityworker stdout | 2024-06-14 05:22:58,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:22:58,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:22:58,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:22:58,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:22:58,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:22:58,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:22:58,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:22:58,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:22:58,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:22:58,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:22:58,210 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:22:58,210 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:22:58,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 17, 58, 204005), 1, 2])
securityworker stdout | 2024-06-14 05:22:58,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:22:58,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:22:58,212 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 17, 58, 204005), 1, 2])
securityworker stdout | 2024-06-14 05:22:58,213 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:22:58,213 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:22:58,213 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:22:58,213 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:22:58,213 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:22:58,213 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:22:58,214 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:22:58,214 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:22:58,214 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:22:58,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:28 UTC)" executed successfully
autopruneworker stdout | 2024-06-14 05:22:58,269 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityscanningnotificationworker stdout | 2024-06-14 05:22:59,194 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:22:59,423 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:22:59,427 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:22:59,431 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:23:00,315 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:23:00,318 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:23:00,321 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:23:00,325 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:23:00,329 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:23:00,333 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:23:00,335 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:23:00,338 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:23:00,341 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
notificationworker stdout | 2024-06-14 05:23:01,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:23:01,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:11.591084+00:00 (in 9.999425 seconds)
notificationworker stdout | 2024-06-14 05:23:01,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:11 UTC)" (scheduled at 2024-06-14 05:23:01.591084+00:00)
notificationworker stdout | 2024-06-14 05:23:01,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:23:01,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 1, 592175), True, datetime.datetime(2024, 6, 14, 5, 23, 1, 592175), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:23:01,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:23:01,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:23:01,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:11 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:23:01,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:23:01,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:31.891735+00:00 (in 29.999405 seconds)
buildlogsarchiver stdout | 2024-06-14 05:23:01,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:31 UTC)" (scheduled at 2024-06-14 05:23:01.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:23:01,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 23, 1, 892732), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:23:01,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:23:01,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:23:01,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:31 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:23:02,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:23:02,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:32.991899+00:00 (in 29.999500 seconds)
gcworker stdout | 2024-06-14 05:23:02,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:32 UTC)" (scheduled at 2024-06-14 05:23:02.991899+00:00)
gcworker stdout | 2024-06-14 05:23:02,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717132982992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:23:03,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:23:03,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:23:03,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:32 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:23:03,798 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:23:03,799 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:15.800790+00:00 (in 12.001539 seconds)
namespacegcworker stdout | 2024-06-14 05:23:03,799 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:03 UTC)" (scheduled at 2024-06-14 05:23:03.798679+00:00)
namespacegcworker stdout | 2024-06-14 05:23:03,799 [75] [DEBUG] [workers.queueworker] Getting work item from queue.
namespacegcworker stdout | 2024-06-14 05:23:03,800 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 3, 799572), True, datetime.datetime(2024, 6, 14, 5, 23, 3, 799572), 0, 'namespacegc/%', 50, 1, 0])
namespacegcworker stdout | 2024-06-14 05:23:03,806 [75] [DEBUG] [workers.queueworker] No more work.
namespacegcworker stdout | 2024-06-14 05:23:03,806 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:23:03,807 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:03 UTC)" executed successfully
globalpromstats stdout | 2024-06-14 05:23:04,495 [68] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
globalpromstats stdout | 2024-06-14 05:23:04,495 [68] [INFO] [apscheduler.executors.default] Running job "GlobalPrometheusStatsWorker._try_report_stats (trigger: interval[1:00:00], next run at: 2024-06-14 05:23:04 UTC)" (scheduled at 2024-06-14 05:23:04.495040+00:00)
globalpromstats stdout | 2024-06-14 05:23:04,496 [68] [DEBUG] [__main__] Attempting to report stats
globalpromstats stdout | 2024-06-14 05:23:04,496 [68] [DEBUG] [util.locking] Acquiring global lock GLOBAL_PROM_STATS
globalpromstats stdout | 2024-06-14 05:23:04,496 [68] [DEBUG] [redis_lock.acquire] Getting 'lock:GLOBAL_PROM_STATS' ...
globalpromstats stdout | 2024-06-14 05:23:04,496 [68] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 06:23:04.495040+00:00 (in 3599.998405 seconds)
globalpromstats stdout | 2024-06-14 05:23:04,496 [68] [INFO] [redis_lock.acquire] Got lock for 'lock:GLOBAL_PROM_STATS'.
globalpromstats stdout | 2024-06-14 05:23:04,497 [68] [DEBUG] [util.locking] Acquired lock GLOBAL_PROM_STATS
globalpromstats stdout | 2024-06-14 05:23:04,497 [68] [DEBUG] [__main__] Reporting global stats
globalpromstats stdout | 2024-06-14 05:23:04,497 [68] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "repository" AS "t1") AS "_wrapped"', [])
globalpromstats stdout | 2024-06-14 05:23:04,503 [68] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ((("t1"."organization" = %s) AND ("t1"."robot" = %s)) AND ("t1"."id" NOT IN (SELECT "t2"."namespace_id" FROM "deletednamespace" AS "t2")))) AS "_wrapped"', [False, False])
globalpromstats stdout | 2024-06-14 05:23:04,505 [68] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "user" AS "t1" WHERE ((("t1"."organization" = %s) AND ("t1"."robot" = %s)) AND ("t1"."enabled" = %s))) AS "_wrapped"', [True, False, True])
globalpromstats stdout | 2024-06-14 05:23:04,506 [68] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT 1 FROM "robotaccounttoken" AS "t1") AS "_wrapped"', [])
globalpromstats stdout | 2024-06-14 05:23:04,506 [68] [DEBUG] [data.database] Disconnecting from database.
globalpromstats stdout | 2024-06-14 05:23:04,506 [68] [DEBUG] [util.locking] Releasing lock GLOBAL_PROM_STATS
globalpromstats stdout | 2024-06-14 05:23:04,506 [68] [DEBUG] [redis_lock.release] Releasing 'lock:GLOBAL_PROM_STATS'.
globalpromstats stdout | 2024-06-14 05:23:04,507 [68] [DEBUG] [util.locking] Released lock GLOBAL_PROM_STATS
globalpromstats stdout | 2024-06-14 05:23:04,507 [68] [INFO] [apscheduler.executors.default] Job "GlobalPrometheusStatsWorker._try_report_stats (trigger: interval[1:00:00], next run at: 2024-06-14 06:23:04 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:23:05,378 [271] [DEBUG] [app] Starting request: urn:request:cbfcde28-b52b-4bfd-9f98-6451db7559d5 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:23:05,379 [269] [DEBUG] [app] Starting request: urn:request:f2f0c26a-cd46-4594-aced-71f220cd731c (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:23:05,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:05,381 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:23:05,382 [280] [DEBUG] [app] Starting request: urn:request:58a3464b-dbcf-4a88-b2f7-f4587d4e3eeb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:23:05,383 [287] [DEBUG] [app] Starting request: urn:request:378dd348-77c7-4754-8ee3-b79975e52bf7 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:23:05,383 [280] [DEBUG] [app] Ending request: urn:request:58a3464b-dbcf-4a88-b2f7-f4587d4e3eeb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:58a3464b-dbcf-4a88-b2f7-f4587d4e3eeb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:23:05,383 [287] [DEBUG] [app] Ending request: urn:request:378dd348-77c7-4754-8ee3-b79975e52bf7 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:378dd348-77c7-4754-8ee3-b79975e52bf7', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:23:05,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:23:05,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:05,383 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-web stdout | 2024-06-14 05:23:05,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:05,385 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:05,385 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:05,386 [269] [DEBUG] [app] Starting request: urn:request:8cee82b6-4ad8-40b3-b561-0c66031bfb41 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:23:05,386 [269] [DEBUG] [app] Ending request: urn:request:8cee82b6-4ad8-40b3-b561-0c66031bfb41 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:8cee82b6-4ad8-40b3-b561-0c66031bfb41', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:23:05,387 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.001)
gunicorn-web stdout | 2024-06-14 05:23:05,387 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:05,387 [270] [DEBUG] [app] Starting request: urn:request:30adc65f-13a1-43ca-b08b-bc6f2d489231 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:23:05,387 [270] [DEBUG] [app] Ending request: urn:request:30adc65f-13a1-43ca-b08b-bc6f2d489231 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:30adc65f-13a1-43ca-b08b-bc6f2d489231', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:23:05,387 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:23:05,388 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:05,388 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:23:05,388 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:23:05,392 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:23:05,392 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:23:05,397 [269] [DEBUG] [app] Ending request: urn:request:f2f0c26a-cd46-4594-aced-71f220cd731c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:f2f0c26a-cd46-4594-aced-71f220cd731c', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:23:05,397 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:23:05,397 [271] [DEBUG] [app] Ending request: urn:request:cbfcde28-b52b-4bfd-9f98-6451db7559d5 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cbfcde28-b52b-4bfd-9f98-6451db7559d5', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:23:05,397 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:23:05,397 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:23:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:23:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.020)
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:23:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
gunicorn-web stdout | 2024-06-14 05:23:05,398 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:23:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
exportactionlogsworker stdout | 2024-06-14 05:23:06,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:23:06,198 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:23.197203+00:00 (in 16.998226 seconds)
exportactionlogsworker stdout | 2024-06-14 05:23:06,199 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:06 UTC)" (scheduled at 2024-06-14 05:23:06.198336+00:00)
exportactionlogsworker stdout | 2024-06-14 05:23:06,199 [66] [DEBUG] [workers.queueworker] Running watchdog.
exportactionlogsworker stdout | 2024-06-14 05:23:06,199 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:06 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:23:07,198 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:23:07,199 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:07.198655+00:00 (in 59.999470 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:23:07,199 [87] [INFO] [apscheduler.executors.default] Running job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:07 UTC)" (scheduled at 2024-06-14 05:23:07.198655+00:00)
quotaregistrysizeworker stdout | 2024-06-14 05:23:07,199 [87] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."size_bytes", "t1"."running", "t1"."queued", "t1"."completed_ms" FROM "quotaregistrysize" AS "t1" LIMIT %s OFFSET %s', [1, 0])
quotaregistrysizeworker stdout | 2024-06-14 05:23:07,205 [87] [DEBUG] [data.database] Disconnecting from database.
quotaregistrysizeworker stdout | 2024-06-14 05:23:07,205 [87] [INFO] [apscheduler.executors.default] Job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:07 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:23:08,779 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:23:09,462 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:23:11,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:23:11,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:21.591084+00:00 (in 9.999442 seconds)
notificationworker stdout | 2024-06-14 05:23:11,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:21 UTC)" (scheduled at 2024-06-14 05:23:11.591084+00:00)
notificationworker stdout | 2024-06-14 05:23:11,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:23:11,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 11, 591983), True, datetime.datetime(2024, 6, 14, 5, 23, 11, 591983), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:23:11,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:23:11,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:23:11,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:21 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:23:14,216 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:23:15,515 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
namespacegcworker stdout | 2024-06-14 05:23:15,801 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:23:15,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:03.798679+00:00 (in 47.997239 seconds)
namespacegcworker stdout | 2024-06-14 05:23:15,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:15 UTC)" (scheduled at 2024-06-14 05:23:15.800790+00:00)
namespacegcworker stdout | 2024-06-14 05:23:15,801 [75] [DEBUG] [workers.queueworker] Running watchdog.
namespacegcworker stdout | 2024-06-14 05:23:15,801 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:15 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:23:16,072 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:23:16,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:23:16,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:28.194620+00:00 (in 12.002528 seconds)
securityworker stdout | 2024-06-14 05:23:16,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:46 UTC)" (scheduled at 2024-06-14 05:23:16.191483+00:00)
securityworker stdout | 2024-06-14 05:23:16,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:23:16,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:23:16,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:23:16,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:23:16,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:23:16,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:23:16,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:46 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:23:16,728 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:23:17,297 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:23:18,546 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:23:18,781 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:23:19,102 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:23:19,107 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:23:19,110 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:23:19,114 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:23:19,117 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:23:20,379 [268] [DEBUG] [app] Starting request: urn:request:4926bc4c-2577-4c0d-876d-d4959141053d (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:23:20,379 [271] [DEBUG] [app] Starting request: urn:request:5fd3e40b-1f58-45d0-bd7c-9d407b5aa5d2 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:23:20,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:20,381 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:23:20,383 [286] [DEBUG] [app] Starting request: urn:request:71697d7a-ce8b-4079-a6f9-73293bbe6503 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:23:20,383 [287] [DEBUG] [app] Starting request: urn:request:70bb5f12-2542-4329-b8ef-3e2944317d89 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:23:20,383 [286] [DEBUG] [app] Ending request: urn:request:71697d7a-ce8b-4079-a6f9-73293bbe6503 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:71697d7a-ce8b-4079-a6f9-73293bbe6503', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:23:20,383 [287] [DEBUG] [app] Ending request: urn:request:70bb5f12-2542-4329-b8ef-3e2944317d89 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:70bb5f12-2542-4329-b8ef-3e2944317d89', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-registry stdout | 2024-06-14 05:23:20,383 [286] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:23:20,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:20,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:20,384 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:20,385 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:20,386 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:20,386 [268] [DEBUG] [app] Starting request: urn:request:4ab85935-4d73-4de0-a4fc-e73be138d7bb (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:23:20,387 [268] [DEBUG] [app] Ending request: urn:request:4ab85935-4d73-4de0-a4fc-e73be138d7bb (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4ab85935-4d73-4de0-a4fc-e73be138d7bb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:23:20,387 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:20,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:20,387 [270] [DEBUG] [app] Starting request: urn:request:3e943aa6-7d9d-40ca-9a50-f793ec45a025 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:23:20,388 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:23:20,388 [270] [DEBUG] [app] Ending request: urn:request:3e943aa6-7d9d-40ca-9a50-f793ec45a025 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:3e943aa6-7d9d-40ca-9a50-f793ec45a025', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:23:20,388 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.002)
gunicorn-web stdout | 2024-06-14 05:23:20,388 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:20,389 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:23:20,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:23:20,393 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:23:20,396 [268] [DEBUG] [app] Ending request: urn:request:4926bc4c-2577-4c0d-876d-d4959141053d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:4926bc4c-2577-4c0d-876d-d4959141053d', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:23:20,396 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:23:20,396 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:23:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:23:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:23:20,397 [271] [DEBUG] [app] Ending request: urn:request:5fd3e40b-1f58-45d0-bd7c-9d407b5aa5d2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:5fd3e40b-1f58-45d0-bd7c-9d407b5aa5d2', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:23:20,397 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:23:20,397 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:23:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:23:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
notificationworker stdout | 2024-06-14 05:23:21,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:23:21,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:21.592513+00:00 (in 0.000865 seconds)
notificationworker stdout | 2024-06-14 05:23:21,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:31 UTC)" (scheduled at 2024-06-14 05:23:21.591084+00:00)
notificationworker stdout | 2024-06-14 05:23:21,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:23:21,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 21, 592158), True, datetime.datetime(2024, 6, 14, 5, 23, 21, 592158), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:23:21,593 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:23:21,593 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:31.591084+00:00 (in 9.997146 seconds)
notificationworker stdout | 2024-06-14 05:23:21,594 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:21 UTC)" (scheduled at 2024-06-14 05:23:21.592513+00:00)
notificationworker stdout | 2024-06-14 05:23:21,594 [82] [DEBUG] [workers.queueworker] Running watchdog.
notificationworker stdout | 2024-06-14 05:23:21,594 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:21 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:23:21,600 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:23:21,600 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:23:21,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:31 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:23:22,179 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:23:22,493 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:23:22,920 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:23:23,033 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
exportactionlogsworker stdout | 2024-06-14 05:23:23,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:23:23,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:06.198336+00:00 (in 43.000555 seconds)
exportactionlogsworker stdout | 2024-06-14 05:23:23,197 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:23 UTC)" (scheduled at 2024-06-14 05:23:23.197203+00:00)
exportactionlogsworker stdout | 2024-06-14 05:23:23,198 [66] [DEBUG] [workers.queueworker] Getting work item from queue.
exportactionlogsworker stdout | 2024-06-14 05:23:23,199 [66] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 23, 198171), True, datetime.datetime(2024, 6, 14, 5, 23, 23, 198171), 0, 'exportactionlogs/%', 50, 1, 0])
exportactionlogsworker stdout | 2024-06-14 05:23:23,205 [66] [DEBUG] [workers.queueworker] No more work.
exportactionlogsworker stdout | 2024-06-14 05:23:23,205 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:23:23,205 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:23 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:23:25,154 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:23:25,410 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
autopruneworker stdout | 2024-06-14 05:23:25,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:23:25,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:55.804090+00:00 (in 29.999336 seconds)
autopruneworker stdout | 2024-06-14 05:23:25,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:55 UTC)" (scheduled at 2024-06-14 05:23:25.804090+00:00)
autopruneworker stdout | 2024-06-14 05:23:25,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342545808, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:23:25,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:23:25,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:23:25,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:55 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:23:25,813 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:23:25,957 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:23:27,209 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:23:27,888 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:23:27,948 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
securityworker stdout | 2024-06-14 05:23:28,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:23:28,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:46.191483+00:00 (in 17.996268 seconds)
securityworker stdout | 2024-06-14 05:23:28,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:58 UTC)" (scheduled at 2024-06-14 05:23:28.194620+00:00)
securityworker stdout | 2024-06-14 05:23:28,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:23:28,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:23:28,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:23:28,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:23:28,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:23:28,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:23:28,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:23:28,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:23:28,203 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:23:28,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:23:28,210 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:23:28,210 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:23:28,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 18, 28, 203691), 1, 2])
securityworker stdout | 2024-06-14 05:23:28,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:23:28,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:23:28,212 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 18, 28, 203691), 1, 2])
securityworker stdout | 2024-06-14 05:23:28,213 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:23:28,213 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:23:28,213 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:23:28,213 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:23:28,213 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:23:28,213 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:23:28,213 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:23:28,213 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:23:28,213 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:23:28,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:23:58 UTC)" executed successfully
autopruneworker stdout | 2024-06-14 05:23:28,305 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityscanningnotificationworker stdout | 2024-06-14 05:23:29,216 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:23:29,431 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:23:29,435 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:23:29,438 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:23:30,328 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:23:30,331 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:23:30,334 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:23:30,338 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:23:30,341 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:23:30,344 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:23:30,347 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:23:30,351 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:23:30,353 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
securityscanningnotificationworker stdout | 2024-06-14 05:23:31,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:23:31,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:35.332183+00:00 (in 4.000375 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:23:31,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:31 UTC)" (scheduled at 2024-06-14 05:23:31.331172+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:23:31,332 [110] [DEBUG] [workers.queueworker] Getting work item from queue.
securityscanningnotificationworker stdout | 2024-06-14 05:23:31,333 [110] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 31, 332216), True, datetime.datetime(2024, 6, 14, 5, 23, 31, 332216), 0, 'secscanv4/%', 50, 1, 0])
securityscanningnotificationworker stdout | 2024-06-14 05:23:31,339 [110] [DEBUG] [workers.queueworker] No more work.
securityscanningnotificationworker stdout | 2024-06-14 05:23:31,339 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:23:31,339 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:31 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:23:31,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:23:31,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:41.591084+00:00 (in 9.999476 seconds)
notificationworker stdout | 2024-06-14 05:23:31,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:41 UTC)" (scheduled at 2024-06-14 05:23:31.591084+00:00)
notificationworker stdout | 2024-06-14 05:23:31,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:23:31,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 31, 592118), True, datetime.datetime(2024, 6, 14, 5, 23, 31, 592118), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:23:31,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:23:31,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:23:31,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:41 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:23:31,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:23:31,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:01.891735+00:00 (in 29.999295 seconds)
buildlogsarchiver stdout | 2024-06-14 05:23:31,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:01 UTC)" (scheduled at 2024-06-14 05:23:31.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:23:31,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 23, 31, 892804), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:23:31,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:23:31,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:23:31,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:01 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:23:32,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:23:32,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:02.991899+00:00 (in 29.999486 seconds)
gcworker stdout | 2024-06-14 05:23:32,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:02 UTC)" (scheduled at 2024-06-14 05:23:32.991899+00:00)
gcworker stdout | 2024-06-14 05:23:32,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717133012992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:23:33,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:23:33,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:23:33,000 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:02 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:23:34,997 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:23:34,997 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:42.089819+00:00 (in 7.092334 seconds)
repositorygcworker stdout | 2024-06-14 05:23:34,997 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:34 UTC)" (scheduled at 2024-06-14 05:23:34.996948+00:00)
repositorygcworker stdout | 2024-06-14 05:23:34,997 [109] [DEBUG] [workers.queueworker] Getting work item from queue.
repositorygcworker stdout | 2024-06-14 05:23:34,998 [109] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 34, 997896), True, datetime.datetime(2024, 6, 14, 5, 23, 34, 997896), 0, 'repositorygc/%', 50, 1, 0])
repositorygcworker stdout | 2024-06-14 05:23:35,005 [109] [DEBUG] [workers.queueworker] No more work.
repositorygcworker stdout | 2024-06-14 05:23:35,005 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:23:35,005 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:34 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:23:35,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:23:35,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:31.331172+00:00 (in 55.998443 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:23:35,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:35 UTC)" (scheduled at 2024-06-14 05:23:35.332183+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:23:35,333 [110] [DEBUG] [workers.queueworker] Running watchdog.
securityscanningnotificationworker stdout | 2024-06-14 05:23:35,333 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:35 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:23:35,378 [271] [DEBUG] [app] Starting request: urn:request:0ed8ed2c-fddb-4b15-bb67-7721e4875ba8 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:23:35,378 [268] [DEBUG] [app] Starting request: urn:request:9a71db9b-d73c-4fae-8547-8769217a7725 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:23:35,379 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:35,379 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:23:35,382 [280] [DEBUG] [app] Starting request: urn:request:f9593735-d0e2-4591-9e4e-a7c546ca4c31 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:23:35,382 [280] [DEBUG] [app] Ending request: urn:request:f9593735-d0e2-4591-9e4e-a7c546ca4c31 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f9593735-d0e2-4591-9e4e-a7c546ca4c31', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.003 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:23:35,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:35,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:23:35,383 [284] [DEBUG] [app] Starting request: urn:request:a1ace342-3ba1-4f95-9566-0cd90afd622f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:23:35,384 [284] [DEBUG] [app] Ending request: urn:request:a1ace342-3ba1-4f95-9566-0cd90afd622f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a1ace342-3ba1-4f95-9566-0cd90afd622f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:23:35,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.003 162 0.004)
gunicorn-registry stdout | 2024-06-14 05:23:35,385 [284] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:35,385 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:35,385 [271] [DEBUG] [app] Starting request: urn:request:0c5daf67-d887-48d9-b50f-e23aa861c519 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:23:35,385 [271] [DEBUG] [app] Ending request: urn:request:0c5daf67-d887-48d9-b50f-e23aa861c519 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:0c5daf67-d887-48d9-b50f-e23aa861c519', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:23:35,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:35,386 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:35,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:35,386 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:23:35,387 [268] [DEBUG] [app] Starting request: urn:request:b768ac84-d27c-4002-88fd-32a396fcba04 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:23:35,388 [268] [DEBUG] [app] Ending request: urn:request:b768ac84-d27c-4002-88fd-32a396fcba04 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:b768ac84-d27c-4002-88fd-32a396fcba04', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:23:35,388 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:23:35,388 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:35,389 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:23:35,390 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:23:35,393 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:23:35,395 [271] [DEBUG] [app] Ending request: urn:request:0ed8ed2c-fddb-4b15-bb67-7721e4875ba8 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:0ed8ed2c-fddb-4b15-bb67-7721e4875ba8', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:23:35,395 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:23:35,395 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:23:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:23:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:23:35,397 [268] [DEBUG] [app] Ending request: urn:request:9a71db9b-d73c-4fae-8547-8769217a7725 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9a71db9b-d73c-4fae-8547-8769217a7725', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:23:35,397 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:23:35,398 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:23:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:23:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
chunkcleanupworker stdout | 2024-06-14 05:23:38,793 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:23:39,498 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:23:41,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:23:41,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:51.591084+00:00 (in 9.999421 seconds)
notificationworker stdout | 2024-06-14 05:23:41,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:51 UTC)" (scheduled at 2024-06-14 05:23:41.591084+00:00)
notificationworker stdout | 2024-06-14 05:23:41,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:23:41,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 41, 592139), True, datetime.datetime(2024, 6, 14, 5, 23, 41, 592139), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:23:41,600 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:23:41,600 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:23:41,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:23:51 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:23:42,090 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:23:42,090 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:34.996948+00:00 (in 52.906637 seconds)
repositorygcworker stdout | 2024-06-14 05:23:42,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:42 UTC)" (scheduled at 2024-06-14 05:23:42.089819+00:00)
repositorygcworker stdout | 2024-06-14 05:23:42,090 [109] [DEBUG] [workers.queueworker] Running watchdog.
repositorygcworker stdout | 2024-06-14 05:23:42,090 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:24:42 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:23:44,241 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:23:45,526 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
repositorygcworker stdout | 2024-06-14 05:23:46,089 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:23:46,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:23:46,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:23:58.194620+00:00 (in 12.002567 seconds)
securityworker stdout | 2024-06-14 05:23:46,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:16 UTC)" (scheduled at 2024-06-14 05:23:46.191483+00:00)
securityworker stdout | 2024-06-14 05:23:46,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:23:46,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:23:46,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:23:46,196 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:23:46,201 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:23:46,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:23:46,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:16 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:23:46,765 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:23:47,334 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:23:48,579 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:23:48,818 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:23:49,112 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:23:49,117 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:23:49,120 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:23:49,122 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:23:49,125 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:23:50,378 [271] [DEBUG] [app] Starting request: urn:request:550604c5-94d5-4366-9364-1bbdd48a0832 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:23:50,378 [270] [DEBUG] [app] Starting request: urn:request:8b841493-b93b-4d9a-8b61-e6cbfc208b23 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:23:50,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:50,380 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:23:50,382 [280] [DEBUG] [app] Starting request: urn:request:84b85b39-d2c2-4b98-960d-e49ca619d729 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:23:50,382 [280] [DEBUG] [app] Ending request: urn:request:84b85b39-d2c2-4b98-960d-e49ca619d729 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:84b85b39-d2c2-4b98-960d-e49ca619d729', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.002)
gunicorn-registry stdout | 2024-06-14 05:23:50,382 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:50,382 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:23:50,383 [282] [DEBUG] [app] Starting request: urn:request:5417570b-870e-4218-8b50-bb290e31e2c2 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:23:50,384 [282] [DEBUG] [app] Ending request: urn:request:5417570b-870e-4218-8b50-bb290e31e2c2 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5417570b-870e-4218-8b50-bb290e31e2c2', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:23:50,384 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.003 162 0.003)
gunicorn-registry stdout | 2024-06-14 05:23:50,384 [282] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:50,384 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:50,385 [270] [DEBUG] [app] Starting request: urn:request:009616f5-6563-4a5a-8db6-d5f3d16df733 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:23:50,385 [270] [DEBUG] [app] Ending request: urn:request:009616f5-6563-4a5a-8db6-d5f3d16df733 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:009616f5-6563-4a5a-8db6-d5f3d16df733', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:23:50,385 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:50,386 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:50,386 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:23:50,386 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:23:50,388 [271] [DEBUG] [app] Starting request: urn:request:e87a7a07-ff47-406c-89e5-96a1c5fbd766 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:23:50,388 [271] [DEBUG] [app] Ending request: urn:request:e87a7a07-ff47-406c-89e5-96a1c5fbd766 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:e87a7a07-ff47-406c-89e5-96a1c5fbd766', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:23:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:23:50,389 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:23:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:23:50,389 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:23:50,389 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:23:50,390 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:23:50,393 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:23:50,395 [270] [DEBUG] [app] Ending request: urn:request:8b841493-b93b-4d9a-8b61-e6cbfc208b23 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8b841493-b93b-4d9a-8b61-e6cbfc208b23', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:23:50,395 [270] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:23:50,395 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:23:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:23:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:23:50,397 [271] [DEBUG] [app] Ending request: urn:request:550604c5-94d5-4366-9364-1bbdd48a0832 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:550604c5-94d5-4366-9364-1bbdd48a0832', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:23:50,398 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:23:50,398 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:23:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:23:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.021 118 0.021)
notificationworker stdout | 2024-06-14 05:23:51,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:23:51,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:01.591084+00:00 (in 9.999442 seconds)
notificationworker stdout | 2024-06-14 05:23:51,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:01 UTC)" (scheduled at 2024-06-14 05:23:51.591084+00:00)
notificationworker stdout | 2024-06-14 05:23:51,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:23:51,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 23, 51, 592060), True, datetime.datetime(2024, 6, 14, 5, 23, 51, 592060), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:23:51,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:23:51,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:23:51,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:01 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:23:52,215 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:23:52,513 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:23:52,952 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:23:53,056 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
quotaregistrysizeworker stdout | 2024-06-14 05:23:55,182 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:23:55,446 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
autopruneworker stdout | 2024-06-14 05:23:55,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:23:55,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:25.804090+00:00 (in 29.999391 seconds)
autopruneworker stdout | 2024-06-14 05:23:55,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:25 UTC)" (scheduled at 2024-06-14 05:23:55.804090+00:00)
autopruneworker stdout | 2024-06-14 05:23:55,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342575808, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:23:55,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:23:55,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:23:55,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:25 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:23:55,843 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:23:55,981 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:23:57,225 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:23:57,924 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:23:57,985 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
securityworker stdout | 2024-06-14 05:23:58,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:23:58,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:16.191483+00:00 (in 17.996267 seconds)
securityworker stdout | 2024-06-14 05:23:58,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:28 UTC)" (scheduled at 2024-06-14 05:23:58.194620+00:00)
securityworker stdout | 2024-06-14 05:23:58,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:23:58,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:23:58,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:23:58,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:23:58,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:23:58,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:23:58,198 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:23:58,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:23:58,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:23:58,209 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:23:58,210 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:23:58,210 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:23:58,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 18, 58, 204005), 1, 2])
securityworker stdout | 2024-06-14 05:23:58,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:23:58,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:23:58,212 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 18, 58, 204005), 1, 2])
securityworker stdout | 2024-06-14 05:23:58,213 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:23:58,213 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:23:58,213 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:23:58,213 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:23:58,213 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:23:58,214 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:23:58,214 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:23:58,214 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:23:58,214 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:23:58,214 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:28 UTC)" executed successfully
autopruneworker stdout | 2024-06-14 05:23:58,342 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityscanningnotificationworker stdout | 2024-06-14 05:23:59,253 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:23:59,438 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:23:59,444 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:23:59,447 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:24:00,341 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:24:00,344 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:24:00,346 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:24:00,350 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:24:00,355 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:24:00,359 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:24:00,362 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:24:00,364 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:24:00,367 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
notificationworker stdout | 2024-06-14 05:24:01,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:24:01,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:11.591084+00:00 (in 9.999412 seconds)
notificationworker stdout | 2024-06-14 05:24:01,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:11 UTC)" (scheduled at 2024-06-14 05:24:01.591084+00:00)
notificationworker stdout | 2024-06-14 05:24:01,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:24:01,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 1, 592163), True, datetime.datetime(2024, 6, 14, 5, 24, 1, 592163), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:24:01,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:24:01,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:24:01,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:11 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:24:01,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:24:01,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:31.891735+00:00 (in 29.999383 seconds)
buildlogsarchiver stdout | 2024-06-14 05:24:01,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:31 UTC)" (scheduled at 2024-06-14 05:24:01.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:24:01,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 24, 1, 892714), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:24:01,899 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:24:01,900 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:24:01,900 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:31 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:24:02,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:24:02,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:32.991899+00:00 (in 29.999497 seconds)
gcworker stdout | 2024-06-14 05:24:02,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:32 UTC)" (scheduled at 2024-06-14 05:24:02.991899+00:00)
gcworker stdout | 2024-06-14 05:24:02,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717133042992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:24:03,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:24:03,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:24:03,000 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:32 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:24:03,798 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:24:03,799 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:15.800790+00:00 (in 12.001484 seconds)
namespacegcworker stdout | 2024-06-14 05:24:03,799 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:03 UTC)" (scheduled at 2024-06-14 05:24:03.798679+00:00)
namespacegcworker stdout | 2024-06-14 05:24:03,799 [75] [DEBUG] [workers.queueworker] Getting work item from queue.
namespacegcworker stdout | 2024-06-14 05:24:03,800 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 3, 799624), True, datetime.datetime(2024, 6, 14, 5, 24, 3, 799624), 0, 'namespacegc/%', 50, 1, 0])
namespacegcworker stdout | 2024-06-14 05:24:03,806 [75] [DEBUG] [workers.queueworker] No more work.
namespacegcworker stdout | 2024-06-14 05:24:03,806 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:24:03,806 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:03 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:24:05,378 [268] [DEBUG] [app] Starting request: urn:request:a708e7a5-4fdf-4e24-ad5b-b838bd9cef45 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:24:05,378 [271] [DEBUG] [app] Starting request: urn:request:471d4617-4047-47aa-82d3-4bc675843fe2 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:24:05,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:05,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:24:05,382 [287] [DEBUG] [app] Starting request: urn:request:5fd88c41-6180-41d3-aaab-9c859fbb47ec (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:24:05,382 [284] [DEBUG] [app] Starting request: urn:request:f38cac02-f454-4ed6-a132-934e8d97e91f (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:24:05,382 [287] [DEBUG] [app] Ending request: urn:request:5fd88c41-6180-41d3-aaab-9c859fbb47ec (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5fd88c41-6180-41d3-aaab-9c859fbb47ec', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:24:05,382 [284] [DEBUG] [app] Ending request: urn:request:f38cac02-f454-4ed6-a132-934e8d97e91f (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:f38cac02-f454-4ed6-a132-934e8d97e91f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:24:05,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:24:05,383 [284] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:24:05,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:05,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:05,384 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:05,385 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:05,386 [268] [DEBUG] [app] Starting request: urn:request:87b23c74-5ff5-44a9-a3e5-2aebeae19cc6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:24:05,386 [268] [DEBUG] [app] Ending request: urn:request:87b23c74-5ff5-44a9-a3e5-2aebeae19cc6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:87b23c74-5ff5-44a9-a3e5-2aebeae19cc6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:24:05,386 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:24:05,386 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:05,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:24:05,387 [270] [DEBUG] [app] Starting request: urn:request:375e50ee-1fcd-4451-a667-ba21f12b2ad6 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:24:05,387 [270] [DEBUG] [app] Ending request: urn:request:375e50ee-1fcd-4451-a667-ba21f12b2ad6 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:375e50ee-1fcd-4451-a667-ba21f12b2ad6', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:24:05,388 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:24:05,388 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:05,388 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:24:05,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:24:05,392 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:24:05,395 [268] [DEBUG] [app] Ending request: urn:request:a708e7a5-4fdf-4e24-ad5b-b838bd9cef45 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a708e7a5-4fdf-4e24-ad5b-b838bd9cef45', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:24:05,396 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:24:05,396 [271] [DEBUG] [app] Ending request: urn:request:471d4617-4047-47aa-82d3-4bc675843fe2 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:471d4617-4047-47aa-82d3-4bc675843fe2', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:24:05,396 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:24:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:24:05,396 [271] [DEBUG] [data.database] Disconnecting from database.
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:24:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.019)
gunicorn-web stdout | 2024-06-14 05:24:05,396 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:24:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:24:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.019)
exportactionlogsworker stdout | 2024-06-14 05:24:06,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:24:06,198 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:23.197203+00:00 (in 16.998276 seconds)
exportactionlogsworker stdout | 2024-06-14 05:24:06,199 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:06 UTC)" (scheduled at 2024-06-14 05:24:06.198336+00:00)
exportactionlogsworker stdout | 2024-06-14 05:24:06,199 [66] [DEBUG] [workers.queueworker] Running watchdog.
exportactionlogsworker stdout | 2024-06-14 05:24:06,199 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:06 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:24:07,198 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:24:07,199 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:07.198655+00:00 (in 59.999446 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:24:07,199 [87] [INFO] [apscheduler.executors.default] Running job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:07 UTC)" (scheduled at 2024-06-14 05:24:07.198655+00:00)
quotaregistrysizeworker stdout | 2024-06-14 05:24:07,199 [87] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."size_bytes", "t1"."running", "t1"."queued", "t1"."completed_ms" FROM "quotaregistrysize" AS "t1" LIMIT %s OFFSET %s', [1, 0])
quotaregistrysizeworker stdout | 2024-06-14 05:24:07,205 [87] [DEBUG] [data.database] Disconnecting from database.
quotaregistrysizeworker stdout | 2024-06-14 05:24:07,205 [87] [INFO] [apscheduler.executors.default] Job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:07 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:24:08,830 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:24:09,535 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:24:11,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:24:11,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:21.591084+00:00 (in 9.999339 seconds)
notificationworker stdout | 2024-06-14 05:24:11,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:21 UTC)" (scheduled at 2024-06-14 05:24:11.591084+00:00)
notificationworker stdout | 2024-06-14 05:24:11,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:24:11,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 11, 592297), True, datetime.datetime(2024, 6, 14, 5, 24, 11, 592297), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:24:11,600 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:24:11,600 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:24:11,600 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:21 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:24:14,257 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:24:15,562 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
namespacegcworker stdout | 2024-06-14 05:24:15,801 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:24:15,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:03.798679+00:00 (in 47.997253 seconds)
namespacegcworker stdout | 2024-06-14 05:24:15,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:15 UTC)" (scheduled at 2024-06-14 05:24:15.800790+00:00)
namespacegcworker stdout | 2024-06-14 05:24:15,801 [75] [DEBUG] [workers.queueworker] Running watchdog.
namespacegcworker stdout | 2024-06-14 05:24:15,801 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:15 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:24:16,126 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:24:16,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:24:16,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:28.194620+00:00 (in 12.002584 seconds)
securityworker stdout | 2024-06-14 05:24:16,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:46 UTC)" (scheduled at 2024-06-14 05:24:16.191483+00:00)
securityworker stdout | 2024-06-14 05:24:16,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:24:16,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:24:16,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:24:16,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:24:16,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:24:16,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:24:16,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:46 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:24:16,777 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:24:17,350 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:24:18,601 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:24:18,848 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:24:19,120 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:24:19,125 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:24:19,128 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:24:19,132 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:24:19,136 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:24:20,379 [268] [DEBUG] [app] Starting request: urn:request:3b7b03cc-1cbc-4805-9041-febde9e0452d (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:24:20,379 [271] [DEBUG] [app] Starting request: urn:request:8cac6998-bfd2-4b7c-a018-5679488a240d (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:24:20,380 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:20,381 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:24:20,382 [286] [DEBUG] [app] Starting request: urn:request:eb8edf4c-cc4d-4cf4-93fb-c7dfce8b2d77 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:24:20,383 [280] [DEBUG] [app] Starting request: urn:request:a8534033-51d6-4843-9ef2-5e738a63b8cb (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:24:20,383 [286] [DEBUG] [app] Ending request: urn:request:eb8edf4c-cc4d-4cf4-93fb-c7dfce8b2d77 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:eb8edf4c-cc4d-4cf4-93fb-c7dfce8b2d77', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:24:20,383 [280] [DEBUG] [app] Ending request: urn:request:a8534033-51d6-4843-9ef2-5e738a63b8cb (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a8534033-51d6-4843-9ef2-5e738a63b8cb', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:24:20,383 [286] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-registry stdout | 2024-06-14 05:24:20,383 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:24:20,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:20,384 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:20,385 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:20,385 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:20,387 [270] [DEBUG] [app] Starting request: urn:request:9bcbbe65-0d8e-40e1-b151-2f15b7536ea4 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:24:20,387 [270] [DEBUG] [app] Ending request: urn:request:9bcbbe65-0d8e-40e1-b151-2f15b7536ea4 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9bcbbe65-0d8e-40e1-b151-2f15b7536ea4', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:24:20,387 [271] [DEBUG] [app] Starting request: urn:request:9bb9a8e9-7b51-4d33-bbbc-9f8916805d9c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:24:20,387 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:24:20,387 [271] [DEBUG] [app] Ending request: urn:request:9bb9a8e9-7b51-4d33-bbbc-9f8916805d9c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9bb9a8e9-7b51-4d33-bbbc-9f8916805d9c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.001)
gunicorn-web stdout | 2024-06-14 05:24:20,388 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:24:20,388 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:20,388 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:20,388 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:24:20,388 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:24:20,392 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:24:20,393 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:24:20,396 [271] [DEBUG] [app] Ending request: urn:request:8cac6998-bfd2-4b7c-a018-5679488a240d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8cac6998-bfd2-4b7c-a018-5679488a240d', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:24:20,397 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:24:20,397 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:24:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:24:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
gunicorn-web stdout | 2024-06-14 05:24:20,397 [268] [DEBUG] [app] Ending request: urn:request:3b7b03cc-1cbc-4805-9041-febde9e0452d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3b7b03cc-1cbc-4805-9041-febde9e0452d', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:24:20,397 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:24:20,398 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:24:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:24:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
notificationworker stdout | 2024-06-14 05:24:21,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:24:21,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:21.592513+00:00 (in 0.000920 seconds)
notificationworker stdout | 2024-06-14 05:24:21,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:31 UTC)" (scheduled at 2024-06-14 05:24:21.591084+00:00)
notificationworker stdout | 2024-06-14 05:24:21,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:24:21,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 21, 592034), True, datetime.datetime(2024, 6, 14, 5, 24, 21, 592034), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:24:21,593 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:24:21,593 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:31.591084+00:00 (in 9.997791 seconds)
notificationworker stdout | 2024-06-14 05:24:21,593 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:21 UTC)" (scheduled at 2024-06-14 05:24:21.592513+00:00)
notificationworker stdout | 2024-06-14 05:24:21,593 [82] [DEBUG] [workers.queueworker] Running watchdog.
notificationworker stdout | 2024-06-14 05:24:21,593 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:21 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:24:21,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:24:21,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:24:21,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:31 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:24:22,226 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:24:22,533 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:24:22,963 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:24:23,079 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
exportactionlogsworker stdout | 2024-06-14 05:24:23,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:24:23,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:06.198336+00:00 (in 43.000480 seconds)
exportactionlogsworker stdout | 2024-06-14 05:24:23,198 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:23 UTC)" (scheduled at 2024-06-14 05:24:23.197203+00:00)
exportactionlogsworker stdout | 2024-06-14 05:24:23,198 [66] [DEBUG] [workers.queueworker] Getting work item from queue.
exportactionlogsworker stdout | 2024-06-14 05:24:23,199 [66] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 23, 198284), True, datetime.datetime(2024, 6, 14, 5, 24, 23, 198284), 0, 'exportactionlogs/%', 50, 1, 0])
exportactionlogsworker stdout | 2024-06-14 05:24:23,205 [66] [DEBUG] [workers.queueworker] No more work.
exportactionlogsworker stdout | 2024-06-14 05:24:23,205 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:24:23,205 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:23 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:24:25,219 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:24:25,465 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
autopruneworker stdout | 2024-06-14 05:24:25,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:24:25,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:55.804090+00:00 (in 29.999465 seconds)
autopruneworker stdout | 2024-06-14 05:24:25,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:55 UTC)" (scheduled at 2024-06-14 05:24:25.804090+00:00)
autopruneworker stdout | 2024-06-14 05:24:25,810 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342605809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:24:25,813 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:24:25,813 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:24:25,813 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:55 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:24:25,850 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:24:25,992 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:24:27,241 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:24:27,949 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:24:27,998 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
securityworker stdout | 2024-06-14 05:24:28,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:24:28,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:46.191483+00:00 (in 17.996215 seconds)
securityworker stdout | 2024-06-14 05:24:28,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:58 UTC)" (scheduled at 2024-06-14 05:24:28.194620+00:00)
securityworker stdout | 2024-06-14 05:24:28,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:24:28,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:24:28,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:24:28,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:24:28,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:24:28,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:24:28,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:24:28,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:24:28,203 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:24:28,208 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:24:28,209 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:24:28,210 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:24:28,210 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 19, 28, 203611), 1, 2])
securityworker stdout | 2024-06-14 05:24:28,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:24:28,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:24:28,211 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 19, 28, 203611), 1, 2])
securityworker stdout | 2024-06-14 05:24:28,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:24:28,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:24:28,212 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:24:28,212 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:24:28,212 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:24:28,212 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:24:28,212 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:24:28,213 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:24:28,213 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:24:28,213 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:24:58 UTC)" executed successfully
autopruneworker stdout | 2024-06-14 05:24:28,379 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityscanningnotificationworker stdout | 2024-06-14 05:24:29,286 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:24:29,447 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:24:29,452 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:24:29,456 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:24:30,352 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:24:30,356 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:24:30,359 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:24:30,362 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:24:30,365 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:24:30,370 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:24:30,373 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:24:30,376 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:24:30,379 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
securityscanningnotificationworker stdout | 2024-06-14 05:24:31,331 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:24:31,331 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:35.332183+00:00 (in 4.000378 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:24:31,331 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:31 UTC)" (scheduled at 2024-06-14 05:24:31.331172+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:24:31,332 [110] [DEBUG] [workers.queueworker] Getting work item from queue.
securityscanningnotificationworker stdout | 2024-06-14 05:24:31,333 [110] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 31, 332199), True, datetime.datetime(2024, 6, 14, 5, 24, 31, 332199), 0, 'secscanv4/%', 50, 1, 0])
securityscanningnotificationworker stdout | 2024-06-14 05:24:31,338 [110] [DEBUG] [workers.queueworker] No more work.
securityscanningnotificationworker stdout | 2024-06-14 05:24:31,338 [110] [DEBUG] [data.database] Disconnecting from database.
securityscanningnotificationworker stdout | 2024-06-14 05:24:31,339 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:31 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:24:31,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:24:31,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:36.592214+00:00 (in 5.000632 seconds)
notificationworker stdout | 2024-06-14 05:24:31,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:41 UTC)" (scheduled at 2024-06-14 05:24:31.591084+00:00)
notificationworker stdout | 2024-06-14 05:24:31,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:24:31,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 31, 591878), True, datetime.datetime(2024, 6, 14, 5, 24, 31, 591878), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:24:31,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:24:31,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:24:31,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:41 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:24:31,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:24:31,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:01.891735+00:00 (in 29.999404 seconds)
buildlogsarchiver stdout | 2024-06-14 05:24:31,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:01 UTC)" (scheduled at 2024-06-14 05:24:31.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:24:31,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 24, 31, 892685), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:24:31,898 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:24:31,898 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:24:31,898 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:01 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:24:32,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:24:32,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:02.991899+00:00 (in 29.999474 seconds)
gcworker stdout | 2024-06-14 05:24:32,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:02 UTC)" (scheduled at 2024-06-14 05:24:32.991899+00:00)
gcworker stdout | 2024-06-14 05:24:32,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717133072992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:24:33,000 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:24:33,000 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:24:33,000 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:02 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:24:34,997 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:24:34,997 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:42.089819+00:00 (in 7.092348 seconds)
repositorygcworker stdout | 2024-06-14 05:24:34,997 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:34 UTC)" (scheduled at 2024-06-14 05:24:34.996948+00:00)
repositorygcworker stdout | 2024-06-14 05:24:34,997 [109] [DEBUG] [workers.queueworker] Getting work item from queue.
repositorygcworker stdout | 2024-06-14 05:24:34,999 [109] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 34, 997903), True, datetime.datetime(2024, 6, 14, 5, 24, 34, 997903), 0, 'repositorygc/%', 50, 1, 0])
repositorygcworker stdout | 2024-06-14 05:24:35,004 [109] [DEBUG] [workers.queueworker] No more work.
repositorygcworker stdout | 2024-06-14 05:24:35,004 [109] [DEBUG] [data.database] Disconnecting from database.
repositorygcworker stdout | 2024-06-14 05:24:35,004 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:34 UTC)" executed successfully
securityscanningnotificationworker stdout | 2024-06-14 05:24:35,332 [110] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityscanningnotificationworker stdout | 2024-06-14 05:24:35,332 [110] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:31.331172+00:00 (in 55.998423 seconds)
securityscanningnotificationworker stdout | 2024-06-14 05:24:35,332 [110] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:35 UTC)" (scheduled at 2024-06-14 05:24:35.332183+00:00)
securityscanningnotificationworker stdout | 2024-06-14 05:24:35,333 [110] [DEBUG] [workers.queueworker] Running watchdog.
securityscanningnotificationworker stdout | 2024-06-14 05:24:35,333 [110] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:35 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:24:35,379 [270] [DEBUG] [app] Starting request: urn:request:bec7e626-8430-408a-be00-cdc6f1630250 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:24:35,379 [268] [DEBUG] [app] Starting request: urn:request:3c144cee-1ce4-44a7-8c98-895900f5de70 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:24:35,380 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:35,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:24:35,382 [287] [DEBUG] [app] Starting request: urn:request:ef4a7205-a61b-4280-afbe-bbac335bf4b9 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:24:35,383 [287] [DEBUG] [app] Ending request: urn:request:ef4a7205-a61b-4280-afbe-bbac335bf4b9 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:ef4a7205-a61b-4280-afbe-bbac335bf4b9', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:24:35,383 [280] [DEBUG] [app] Starting request: urn:request:a8ad57f8-9e32-4a48-912b-c58210a32099 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:24:35,383 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:24:35,383 [280] [DEBUG] [app] Ending request: urn:request:a8ad57f8-9e32-4a48-912b-c58210a32099 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a8ad57f8-9e32-4a48-912b-c58210a32099', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
gunicorn-web stdout | 2024-06-14 05:24:35,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-registry stdout | 2024-06-14 05:24:35,384 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:24:35,384 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:35 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.001)
gunicorn-web stdout | 2024-06-14 05:24:35,384 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:35,385 [270] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:35,386 [268] [DEBUG] [app] Starting request: urn:request:24ad6270-fdbb-44d2-ac48-a8e38ff14837 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:24:35,386 [268] [DEBUG] [app] Ending request: urn:request:24ad6270-fdbb-44d2-ac48-a8e38ff14837 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:24ad6270-fdbb-44d2-ac48-a8e38ff14837', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:24:35,386 [268] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:24:35,386 [271] [DEBUG] [app] Starting request: urn:request:9dfdf6dd-a0ee-402b-8c7b-35e77de2f2cc (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:24:35,386 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:35,387 [271] [DEBUG] [app] Ending request: urn:request:9dfdf6dd-a0ee-402b-8c7b-35e77de2f2cc (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9dfdf6dd-a0ee-402b-8c7b-35e77de2f2cc', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:24:35,387 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:24:35,387 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:35 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:24:35,387 [270] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:35 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.002)
gunicorn-web stdout | 2024-06-14 05:24:35,388 [270] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:24:35,391 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:24:35,391 [270] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:24:35,396 [268] [DEBUG] [app] Ending request: urn:request:3c144cee-1ce4-44a7-8c98-895900f5de70 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:3c144cee-1ce4-44a7-8c98-895900f5de70', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:24:35,396 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:24:35,396 [270] [DEBUG] [app] Ending request: urn:request:bec7e626-8430-408a-be00-cdc6f1630250 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:bec7e626-8430-408a-be00-cdc6f1630250', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:24:35,396 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:24:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:24:35,396 [270] [DEBUG] [data.database] Disconnecting from database.
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:24:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:24:35,397 [270] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:24:35 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:24:35 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.018)
notificationworker stdout | 2024-06-14 05:24:36,592 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:24:36,592 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:41.591084+00:00 (in 4.998338 seconds)
notificationworker stdout | 2024-06-14 05:24:36,592 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:29:36 UTC)" (scheduled at 2024-06-14 05:24:36.592214+00:00)
notificationworker stdout | 2024-06-14 05:24:36,593 [82] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2024, 6, 14, 5, 24, 36, 593165), 'notification/%'])
notificationworker stdout | 2024-06-14 05:24:36,599 [82] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 24, 36, 593165), True, datetime.datetime(2024, 6, 14, 5, 24, 36, 593165), 0, 'notification/%'])
notificationworker stdout | 2024-06-14 05:24:36,600 [82] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 24, 36, 593165), True, datetime.datetime(2024, 6, 14, 5, 24, 36, 593165), 0, 'notification/%', False, datetime.datetime(2024, 6, 14, 5, 24, 36, 593165), 'notification/%'])
notificationworker stdout | 2024-06-14 05:24:36,601 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:24:36,602 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:29:36 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:24:38,842 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:24:39,572 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:24:41,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:24:41,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:51.591084+00:00 (in 9.999416 seconds)
notificationworker stdout | 2024-06-14 05:24:41,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:51 UTC)" (scheduled at 2024-06-14 05:24:41.591084+00:00)
notificationworker stdout | 2024-06-14 05:24:41,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:24:41,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 41, 592045), True, datetime.datetime(2024, 6, 14, 5, 24, 41, 592045), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:24:41,599 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:24:41,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:24:41,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:24:51 UTC)" executed successfully
repositorygcworker stdout | 2024-06-14 05:24:42,090 [109] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
repositorygcworker stdout | 2024-06-14 05:24:42,090 [109] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:34.996948+00:00 (in 52.906652 seconds)
repositorygcworker stdout | 2024-06-14 05:24:42,090 [109] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:42 UTC)" (scheduled at 2024-06-14 05:24:42.089819+00:00)
repositorygcworker stdout | 2024-06-14 05:24:42,090 [109] [DEBUG] [workers.queueworker] Running watchdog.
repositorygcworker stdout | 2024-06-14 05:24:42,090 [109] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:25:42 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:24:44,266 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:24:45,599 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
repositorygcworker stdout | 2024-06-14 05:24:46,163 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:24:46,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:24:46,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:24:58.194620+00:00 (in 12.002503 seconds)
securityworker stdout | 2024-06-14 05:24:46,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:16 UTC)" (scheduled at 2024-06-14 05:24:46.191483+00:00)
securityworker stdout | 2024-06-14 05:24:46,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:24:46,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:24:46,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:24:46,197 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:24:46,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:24:46,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:24:46,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:16 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:24:46,786 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:24:47,363 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:24:48,625 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:24:48,884 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:24:49,129 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:24:49,135 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:24:49,139 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:24:49,142 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:24:49,144 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:24:50,379 [271] [DEBUG] [app] Starting request: urn:request:8a5ca7a6-9b6b-4283-9455-7e1df65cd84a (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:24:50,379 [268] [DEBUG] [app] Starting request: urn:request:cc6e595f-2868-40d5-9eee-34b9847e2f81 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:24:50,380 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:50,381 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:24:50,382 [286] [DEBUG] [app] Starting request: urn:request:8c6afa9f-1f7f-4bf6-ad7f-d415801d83e8 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:24:50,382 [282] [DEBUG] [app] Starting request: urn:request:4cdb0f78-2eb4-49a9-a1e3-dbc0cd720a81 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:24:50,383 [286] [DEBUG] [app] Ending request: urn:request:8c6afa9f-1f7f-4bf6-ad7f-d415801d83e8 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:8c6afa9f-1f7f-4bf6-ad7f-d415801d83e8', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:24:50,383 [282] [DEBUG] [app] Ending request: urn:request:4cdb0f78-2eb4-49a9-a1e3-dbc0cd720a81 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:4cdb0f78-2eb4-49a9-a1e3-dbc0cd720a81', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:24:50,383 [286] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:24:50,383 [282] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.002)
gunicorn-web stdout | 2024-06-14 05:24:50,383 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:50,383 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:50,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:50,385 [268] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:24:50,386 [271] [DEBUG] [app] Starting request: urn:request:9d09eaa9-8f04-4630-913d-a924cfb2222f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:24:50,386 [271] [DEBUG] [app] Ending request: urn:request:9d09eaa9-8f04-4630-913d-a924cfb2222f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:9d09eaa9-8f04-4630-913d-a924cfb2222f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:24:50,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:24:50,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:50,387 [270] [DEBUG] [app] Starting request: urn:request:49144a36-7763-4e57-b088-449c14db291f (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:24:50,387 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:24:50,387 [270] [DEBUG] [app] Ending request: urn:request:49144a36-7763-4e57-b088-449c14db291f (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:49144a36-7763-4e57-b088-449c14db291f', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:24:50,387 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:24:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:24:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:24:50,387 [268] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:24:50,388 [268] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:24:50,391 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:24:50,392 [268] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:24:50,395 [271] [DEBUG] [app] Ending request: urn:request:8a5ca7a6-9b6b-4283-9455-7e1df65cd84a (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:8a5ca7a6-9b6b-4283-9455-7e1df65cd84a', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:24:50,395 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:24:50,395 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:24:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:24:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.017 118 0.017)
gunicorn-web stdout | 2024-06-14 05:24:50,397 [268] [DEBUG] [app] Ending request: urn:request:cc6e595f-2868-40d5-9eee-34b9847e2f81 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:cc6e595f-2868-40d5-9eee-34b9847e2f81', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:24:50,397 [268] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:24:50,398 [268] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:24:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:24:50 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
notificationworker stdout | 2024-06-14 05:24:51,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:24:51,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:01.591084+00:00 (in 9.999442 seconds)
notificationworker stdout | 2024-06-14 05:24:51,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:25:01 UTC)" (scheduled at 2024-06-14 05:24:51.591084+00:00)
notificationworker stdout | 2024-06-14 05:24:51,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:24:51,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 24, 51, 592067), True, datetime.datetime(2024, 6, 14, 5, 24, 51, 592067), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:24:51,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:24:51,599 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:24:51,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:25:01 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:24:52,248 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:24:52,566 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:24:52,975 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:24:53,086 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
quotaregistrysizeworker stdout | 2024-06-14 05:24:55,255 [87] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'quotaregistrysizeworker.py', 'pid': '87'}
teamsyncworker stdout | 2024-06-14 05:24:55,483 [114] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'teamsyncworker.py', 'pid': '114'}
autopruneworker stdout | 2024-06-14 05:24:55,804 [59] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
autopruneworker stdout | 2024-06-14 05:24:55,804 [59] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:25.804090+00:00 (in 29.999300 seconds)
autopruneworker stdout | 2024-06-14 05:24:55,804 [59] [INFO] [apscheduler.executors.default] Running job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:25 UTC)" (scheduled at 2024-06-14 05:24:55.804090+00:00)
autopruneworker stdout | 2024-06-14 05:24:55,809 [59] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."namespace_id", "t1"."last_ran_ms", "t1"."status" FROM "autoprunetaskstatus" AS "t1" WHERE (("t1"."namespace_id" NOT IN (SELECT "t2"."id" FROM "user" AS "t2" WHERE (("t2"."enabled" = %s) AND ("t2"."id" = "t1"."namespace_id")))) AND (("t1"."last_ran_ms" < %s) OR ("t1"."last_ran_ms" IS %s))) ORDER BY "t1"."last_ran_ms" ASC NULLS first LIMIT %s OFFSET %s FOR UPDATE SKIP LOCKED', [False, 1718342635809, None, 1, 0])
autopruneworker stdout | 2024-06-14 05:24:55,812 [59] [INFO] [__main__] no autoprune tasks found, exiting...
autopruneworker stdout | 2024-06-14 05:24:55,812 [59] [DEBUG] [data.database] Disconnecting from database.
autopruneworker stdout | 2024-06-14 05:24:55,812 [59] [INFO] [apscheduler.executors.default] Job "AutoPruneWorker.prune (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:25 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:24:55,869 [82] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'notificationworker.py', 'pid': '82'}
globalpromstats stdout | 2024-06-14 05:24:56,029 [68] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'globalpromstats.py', 'pid': '68'}
buildlogsarchiver stdout | 2024-06-14 05:24:57,257 [62] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'buildlogsarchiver.py', 'pid': '62'}
builder stdout | 2024-06-14 05:24:57,973 [61] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'builder.py', 'pid': '61'}
storagereplication stdout | 2024-06-14 05:24:58,012 [113] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'storagereplication.py', 'pid': '113'}
securityworker stdout | 2024-06-14 05:24:58,194 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:24:58,195 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:16.191483+00:00 (in 17.996262 seconds)
securityworker stdout | 2024-06-14 05:24:58,195 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:28 UTC)" (scheduled at 2024-06-14 05:24:58.194620+00:00)
securityworker stdout | 2024-06-14 05:24:58,195 [111] [DEBUG] [util.locking] Acquiring global lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:24:58,195 [111] [DEBUG] [redis_lock.acquire] Getting 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST' ...
securityworker stdout | 2024-06-14 05:24:58,197 [111] [INFO] [redis_lock.acquire] Got lock for 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:24:58,197 [111] [DEBUG] [redis_lock.refresh.start] Starting thread to refresh lock every 200.0 seconds
securityworker stdout | 2024-06-14 05:24:58,197 [111] [DEBUG] [util.locking] Acquired lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:24:58,197 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:24:58,197 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:24:58,202 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:24:58,204 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:24:58,208 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" LEFT OUTER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE ((("t2"."id" IS %s) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [None, 1, 2])
securityworker stdout | 2024-06-14 05:24:58,210 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:24:58,210 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:24:58,210 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((("t2"."index_status" = %s) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-1, datetime.datetime(2024, 6, 14, 5, 19, 58, 203778), 1, 2])
securityworker stdout | 2024-06-14 05:24:58,211 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:24:58,211 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:24:58,212 [111] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."repository_id", "t1"."digest", "t1"."media_type_id", "t1"."manifest_bytes", "t1"."config_media_type", "t1"."layers_compressed_size", "t2"."id", "t2"."manifest_id", "t2"."repository_id", "t2"."index_status", "t2"."error_json", "t2"."last_indexed", "t2"."indexer_hash", "t2"."indexer_version", "t2"."metadata_json" FROM "manifest" AS "t1" INNER JOIN "manifestsecuritystatus" AS "t2" ON ("t2"."manifest_id" = "t1"."id") WHERE (((((("t2"."index_status" != %s) AND ("t2"."index_status" != %s)) AND ("t2"."indexer_hash" != %s)) AND ("t2"."last_indexed" < %s)) AND ("t1"."id" >= %s)) AND ("t1"."id" < %s)) ORDER BY "t1"."id"', [-2, -3, 'ee552dad37417b077db06adae1bc83b5', datetime.datetime(2024, 6, 14, 5, 19, 58, 203778), 1, 2])
securityworker stdout | 2024-06-14 05:24:58,212 [111] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 1-2
securityworker stdout | 2024-06-14 05:24:58,212 [111] [INFO] [util.migrate.allocator] No more work
securityworker stdout | 2024-06-14 05:24:58,212 [111] [DEBUG] [util.locking] Releasing lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:24:58,213 [111] [DEBUG] [redis_lock.refresh.shutdown] Signalling the lock refresher to stop
securityworker stdout | 2024-06-14 05:24:58,213 [111] [DEBUG] [redis_lock.refresh.thread.exit] Exit requested, stopping lock refreshing
securityworker stdout | 2024-06-14 05:24:58,213 [111] [DEBUG] [redis_lock.refresh.exit] Lock refresher has stopped
securityworker stdout | 2024-06-14 05:24:58,213 [111] [DEBUG] [redis_lock.release] Releasing 'lock:SECURITYWORKER_INDEX_RECENT_MANIFEST'.
securityworker stdout | 2024-06-14 05:24:58,213 [111] [DEBUG] [util.locking] Released lock SECURITYWORKER_INDEX_RECENT_MANIFEST
securityworker stdout | 2024-06-14 05:24:58,213 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:24:58,213 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_recent_manifests_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:28 UTC)" executed successfully
autopruneworker stdout | 2024-06-14 05:24:58,416 [59] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'autopruneworker.py', 'pid': '59'}
securityscanningnotificationworker stdout | 2024-06-14 05:24:59,300 [110] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityscanningnotificationworker.py', 'pid': '110'}
gunicorn-secscan stdout | 2024-06-14 05:24:59,455 [70] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '70'}
gunicorn-secscan stdout | 2024-06-14 05:24:59,459 [277] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '277'}
gunicorn-secscan stdout | 2024-06-14 05:24:59,463 [276] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'secscan:application', 'pid': '276'}
gunicorn-registry stdout | 2024-06-14 05:25:00,362 [281] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '281'}
gunicorn-registry stdout | 2024-06-14 05:25:00,366 [287] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '287'}
gunicorn-registry stdout | 2024-06-14 05:25:00,369 [280] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '280'}
gunicorn-registry stdout | 2024-06-14 05:25:00,372 [285] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '285'}
gunicorn-registry stdout | 2024-06-14 05:25:00,376 [69] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '69'}
gunicorn-registry stdout | 2024-06-14 05:25:00,383 [286] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '286'}
gunicorn-registry stdout | 2024-06-14 05:25:00,387 [282] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '282'}
gunicorn-registry stdout | 2024-06-14 05:25:00,390 [283] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '283'}
gunicorn-registry stdout | 2024-06-14 05:25:00,392 [284] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'registry:application', 'pid': '284'}
notificationworker stdout | 2024-06-14 05:25:01,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:25:01,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:11.591084+00:00 (in 9.999414 seconds)
notificationworker stdout | 2024-06-14 05:25:01,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:25:11 UTC)" (scheduled at 2024-06-14 05:25:01.591084+00:00)
notificationworker stdout | 2024-06-14 05:25:01,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:25:01,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 25, 1, 592005), True, datetime.datetime(2024, 6, 14, 5, 25, 1, 592005), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:25:01,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:25:01,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:25:01,599 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:25:11 UTC)" executed successfully
buildlogsarchiver stdout | 2024-06-14 05:25:01,892 [62] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
buildlogsarchiver stdout | 2024-06-14 05:25:01,892 [62] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:31.891735+00:00 (in 29.999376 seconds)
buildlogsarchiver stdout | 2024-06-14 05:25:01,892 [62] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:31 UTC)" (scheduled at 2024-06-14 05:25:01.891735+00:00)
buildlogsarchiver stdout | 2024-06-14 05:25:01,893 [62] [DEBUG] [peewee] ('SELECT "candidates"."id" FROM (SELECT "t1"."id" FROM "repositorybuild" AS "t1" WHERE ((("t1"."phase" IN (%s, %s, %s)) OR ("t1"."started" < %s)) AND ("t1"."logs_archived" = %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', ['complete', 'error', 'cancelled', datetime.datetime(2024, 5, 30, 5, 25, 1, 892749), False, 50, 1, 0])
buildlogsarchiver stdout | 2024-06-14 05:25:01,898 [62] [DEBUG] [__main__] No more builds to archive
buildlogsarchiver stdout | 2024-06-14 05:25:01,899 [62] [DEBUG] [data.database] Disconnecting from database.
buildlogsarchiver stdout | 2024-06-14 05:25:01,899 [62] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:31 UTC)" executed successfully
gcworker stdout | 2024-06-14 05:25:02,992 [67] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
gcworker stdout | 2024-06-14 05:25:02,992 [67] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:32.991899+00:00 (in 29.999445 seconds)
gcworker stdout | 2024-06-14 05:25:02,992 [67] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:32 UTC)" (scheduled at 2024-06-14 05:25:02.991899+00:00)
gcworker stdout | 2024-06-14 05:25:02,993 [67] [DEBUG] [peewee] ('SELECT "candidates"."repository_id" FROM (SELECT DISTINCT "t1"."repository_id" FROM "tag" AS "t1" INNER JOIN "repository" AS "t2" ON ("t1"."repository_id" = "t2"."id") INNER JOIN "user" AS "t3" ON ("t2"."namespace_user_id" = "t3"."id") WHERE ((((NOT ("t1"."lifetime_end_ms" IS %s) AND ("t1"."lifetime_end_ms" <= %s)) AND ("t3"."removed_tag_expiration_s" = %s)) AND ("t3"."enabled" = %s)) AND ("t2"."state" != %s)) LIMIT %s) AS "candidates" ORDER BY Random() LIMIT %s OFFSET %s', [None, 1717133102992, 1209600, True, 3, 500, 1, 0])
gcworker stdout | 2024-06-14 05:25:03,001 [67] [DEBUG] [__main__] No repository with garbage found
gcworker stdout | 2024-06-14 05:25:03,001 [67] [DEBUG] [data.database] Disconnecting from database.
gcworker stdout | 2024-06-14 05:25:03,001 [67] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:32 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:25:03,798 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:25:03,799 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:08.800520+00:00 (in 5.001245 seconds)
namespacegcworker stdout | 2024-06-14 05:25:03,799 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:03 UTC)" (scheduled at 2024-06-14 05:25:03.798679+00:00)
namespacegcworker stdout | 2024-06-14 05:25:03,799 [75] [DEBUG] [workers.queueworker] Getting work item from queue.
namespacegcworker stdout | 2024-06-14 05:25:03,800 [75] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 25, 3, 799558), True, datetime.datetime(2024, 6, 14, 5, 25, 3, 799558), 0, 'namespacegc/%', 50, 1, 0])
namespacegcworker stdout | 2024-06-14 05:25:03,806 [75] [DEBUG] [workers.queueworker] No more work.
namespacegcworker stdout | 2024-06-14 05:25:03,806 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:25:03,806 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:03 UTC)" executed successfully
gunicorn-web stdout | 2024-06-14 05:25:05,378 [269] [DEBUG] [app] Starting request: urn:request:9fe84e04-ec84-4086-9de7-ce2558ed037c (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:25:05,378 [271] [DEBUG] [app] Starting request: urn:request:6cbc01ca-f357-4e42-b9e5-f7b382f77360 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:25:05,379 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:25:05,380 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:25:05,381 [280] [DEBUG] [app] Starting request: urn:request:a5d8c34d-07d2-4ff9-b808-ef889788f4ea (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:25:05,382 [286] [DEBUG] [app] Starting request: urn:request:89c109b9-14ed-4e77-a4da-95bd7c1fe224 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:25:05,382 [280] [DEBUG] [app] Ending request: urn:request:a5d8c34d-07d2-4ff9-b808-ef889788f4ea (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:a5d8c34d-07d2-4ff9-b808-ef889788f4ea', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:25:05,382 [286] [DEBUG] [app] Ending request: urn:request:89c109b9-14ed-4e77-a4da-95bd7c1fe224 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:89c109b9-14ed-4e77-a4da-95bd7c1fe224', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:25:05,382 [280] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:25:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:25:05,382 [286] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:25:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:25:05,382 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:25:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 162 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:25:05 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-web stdout | 2024-06-14 05:25:05,383 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:25:05,384 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:25:05,384 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:25:05,385 [271] [DEBUG] [app] Starting request: urn:request:28e29a36-86b0-41d3-b311-bf8425500a57 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:25:05,385 [271] [DEBUG] [app] Ending request: urn:request:28e29a36-86b0-41d3-b311-bf8425500a57 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:28e29a36-86b0-41d3-b311-bf8425500a57', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:25:05,386 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:25:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:25:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 159 0.001)
gunicorn-web stdout | 2024-06-14 05:25:05,386 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:25:05,386 [270] [DEBUG] [app] Starting request: urn:request:6885462b-09ad-497b-b6d5-dd06f1069742 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:25:05,386 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:25:05,386 [270] [DEBUG] [app] Ending request: urn:request:6885462b-09ad-497b-b6d5-dd06f1069742 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:6885462b-09ad-497b-b6d5-dd06f1069742', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:25:05,387 [270] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:25:05 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:25:05,387 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:25:05 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:25:05,387 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:25:05,390 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:25:05,391 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:25:05,394 [271] [DEBUG] [app] Ending request: urn:request:6cbc01ca-f357-4e42-b9e5-f7b382f77360 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:6cbc01ca-f357-4e42-b9e5-f7b382f77360', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:25:05,395 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:25:05,395 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:25:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:25:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.018 118 0.018)
gunicorn-web stdout | 2024-06-14 05:25:05,395 [269] [DEBUG] [app] Ending request: urn:request:9fe84e04-ec84-4086-9de7-ce2558ed037c (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:9fe84e04-ec84-4086-9de7-ce2558ed037c', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:25:05,396 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:25:05,396 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:25:05 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:25:05 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.019 118 0.019)
exportactionlogsworker stdout | 2024-06-14 05:25:06,198 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:25:06,199 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:23.197203+00:00 (in 16.998195 seconds)
exportactionlogsworker stdout | 2024-06-14 05:25:06,199 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:06 UTC)" (scheduled at 2024-06-14 05:25:06.198336+00:00)
exportactionlogsworker stdout | 2024-06-14 05:25:06,199 [66] [DEBUG] [workers.queueworker] Running watchdog.
exportactionlogsworker stdout | 2024-06-14 05:25:06,199 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:06 UTC)" executed successfully
quotaregistrysizeworker stdout | 2024-06-14 05:25:07,198 [87] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
quotaregistrysizeworker stdout | 2024-06-14 05:25:07,199 [87] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:26:07.198655+00:00 (in 59.999405 seconds)
quotaregistrysizeworker stdout | 2024-06-14 05:25:07,199 [87] [INFO] [apscheduler.executors.default] Running job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:07 UTC)" (scheduled at 2024-06-14 05:25:07.198655+00:00)
quotaregistrysizeworker stdout | 2024-06-14 05:25:07,199 [87] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."size_bytes", "t1"."running", "t1"."queued", "t1"."completed_ms" FROM "quotaregistrysize" AS "t1" LIMIT %s OFFSET %s', [1, 0])
quotaregistrysizeworker stdout | 2024-06-14 05:25:07,204 [87] [DEBUG] [data.database] Disconnecting from database.
quotaregistrysizeworker stdout | 2024-06-14 05:25:07,204 [87] [INFO] [apscheduler.executors.default] Job "QuotaRegistrySizeWorker._calculate_registry_size (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:07 UTC)" executed successfully
namespacegcworker stdout | 2024-06-14 05:25:08,800 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:25:08,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:15.800790+00:00 (in 6.999596 seconds)
namespacegcworker stdout | 2024-06-14 05:25:08,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:30:08 UTC)" (scheduled at 2024-06-14 05:25:08.800520+00:00)
namespacegcworker stdout | 2024-06-14 05:25:08,802 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [False, datetime.datetime(2024, 6, 14, 5, 25, 8, 801467), 'namespacegc/%'])
namespacegcworker stdout | 2024-06-14 05:25:08,807 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 25, 8, 801467), True, datetime.datetime(2024, 6, 14, 5, 25, 8, 801467), 0, 'namespacegc/%'])
namespacegcworker stdout | 2024-06-14 05:25:08,808 [75] [DEBUG] [peewee] ('SELECT COUNT(1) FROM (SELECT DISTINCT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) AND NOT ("t1"."queue_name" IN (SELECT "t1"."queue_name" FROM "queueitem" AS "t1" WHERE ((("t1"."available" = %s) AND ("t1"."processing_expires" > %s)) AND ("t1"."queue_name" ILIKE %s)))))) AS "_wrapped"', [datetime.datetime(2024, 6, 14, 5, 25, 8, 801467), True, datetime.datetime(2024, 6, 14, 5, 25, 8, 801467), 0, 'namespacegc/%', False, datetime.datetime(2024, 6, 14, 5, 25, 8, 801467), 'namespacegc/%'])
namespacegcworker stdout | 2024-06-14 05:25:08,809 [75] [DEBUG] [data.database] Disconnecting from database.
namespacegcworker stdout | 2024-06-14 05:25:08,809 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2024-06-14 05:30:08 UTC)" executed successfully
chunkcleanupworker stdout | 2024-06-14 05:25:08,878 [63] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'chunkcleanupworker.py', 'pid': '63'}
queuecleanupworker stdout | 2024-06-14 05:25:09,609 [84] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'queuecleanupworker.py', 'pid': '84'}
notificationworker stdout | 2024-06-14 05:25:11,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:25:11,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:21.591084+00:00 (in 9.999420 seconds)
notificationworker stdout | 2024-06-14 05:25:11,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:25:21 UTC)" (scheduled at 2024-06-14 05:25:11.591084+00:00)
notificationworker stdout | 2024-06-14 05:25:11,592 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:25:11,593 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 25, 11, 592146), True, datetime.datetime(2024, 6, 14, 5, 25, 11, 592146), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:25:11,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:25:11,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:25:11,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:25:21 UTC)" executed successfully
logrotateworker stdout | 2024-06-14 05:25:14,305 [72] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'logrotateworker.py', 'pid': '72'}
repositoryactioncounter stdout | 2024-06-14 05:25:15,625 [108] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositoryactioncounter.py', 'pid': '108'}
namespacegcworker stdout | 2024-06-14 05:25:15,801 [75] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
namespacegcworker stdout | 2024-06-14 05:25:15,801 [75] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:26:03.798679+00:00 (in 47.997274 seconds)
namespacegcworker stdout | 2024-06-14 05:25:15,801 [75] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:15 UTC)" (scheduled at 2024-06-14 05:25:15.800790+00:00)
namespacegcworker stdout | 2024-06-14 05:25:15,801 [75] [DEBUG] [workers.queueworker] Running watchdog.
namespacegcworker stdout | 2024-06-14 05:25:15,801 [75] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:15 UTC)" executed successfully
securityworker stdout | 2024-06-14 05:25:16,191 [111] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
securityworker stdout | 2024-06-14 05:25:16,192 [111] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:28.194620+00:00 (in 12.002562 seconds)
securityworker stdout | 2024-06-14 05:25:16,192 [111] [INFO] [apscheduler.executors.default] Running job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:46 UTC)" (scheduled at 2024-06-14 05:25:16.191483+00:00)
securityworker stdout | 2024-06-14 05:25:16,192 [111] [DEBUG] [util.secscan.v4.api] generated jwt for security scanner request
securityworker stdout | 2024-06-14 05:25:16,192 [111] [DEBUG] [util.secscan.v4.api] GETing security URL http://quayregistry-clair-app.quay-enterprise.svc.cluster.local/indexer/api/v1/index_state
securityworker stdout | 2024-06-14 05:25:16,195 [111] [DEBUG] [urllib3.connectionpool] http://quayregistry-clair-app.quay-enterprise.svc.cluster.local:80 "GET /indexer/api/v1/index_state HTTP/1.1" 200 None
securityworker stdout | 2024-06-14 05:25:16,196 [111] [DEBUG] [peewee] ('SELECT Max("t1"."id") FROM "manifest" AS "t1"', [])
repositorygcworker stdout | 2024-06-14 05:25:16,197 [109] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'repositorygcworker.py', 'pid': '109'}
securityworker stdout | 2024-06-14 05:25:16,202 [111] [DEBUG] [peewee] ('SELECT Min("t1"."id") FROM "manifest" AS "t1"', [])
securityworker stdout | 2024-06-14 05:25:16,202 [111] [DEBUG] [data.database] Disconnecting from database.
securityworker stdout | 2024-06-14 05:25:16,202 [111] [INFO] [apscheduler.executors.default] Job "SecurityWorker._index_in_scanner (trigger: interval[0:00:30], next run at: 2024-06-14 05:25:46 UTC)" executed successfully
servicekey stdout | 2024-06-14 05:25:16,795 [112] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'servicekeyworker.py', 'pid': '112'}
blobuploadcleanupworker stdout | 2024-06-14 05:25:17,400 [60] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'blobuploadcleanupworker.py', 'pid': '60'}
expiredappspecifictokenworker stdout | 2024-06-14 05:25:18,636 [65] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'expiredappspecifictokenworker.py', 'pid': '65'}
securityworker stdout | 2024-06-14 05:25:18,897 [111] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'securityworker.py', 'pid': '111'}
gunicorn-web stdout | 2024-06-14 05:25:19,138 [271] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '271'}
gunicorn-web stdout | 2024-06-14 05:25:19,144 [268] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '268'}
gunicorn-web stdout | 2024-06-14 05:25:19,148 [71] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '71'}
gunicorn-web stdout | 2024-06-14 05:25:19,152 [270] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '270'}
gunicorn-web stdout | 2024-06-14 05:25:19,155 [269] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'web:application', 'pid': '269'}
gunicorn-web stdout | 2024-06-14 05:25:20,379 [269] [DEBUG] [app] Starting request: urn:request:fb33f04a-75fe-4bea-8d68-82101425ca5d (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:25:20,379 [271] [DEBUG] [app] Starting request: urn:request:a0b36629-27ed-4566-8738-c62cd2b793e6 (/health/instance) {'X-Forwarded-For': '10.129.2.2'}
gunicorn-web stdout | 2024-06-14 05:25:20,381 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:25:20,381 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-registry stdout | 2024-06-14 05:25:20,383 [287] [DEBUG] [app] Starting request: urn:request:b60b9ecd-e280-4472-80e4-12d3ae5176a5 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:25:20,383 [282] [DEBUG] [app] Starting request: urn:request:5761b8a8-cb5a-45a5-868e-7e37347b2841 (/v1/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-registry stdout | 2024-06-14 05:25:20,383 [282] [DEBUG] [app] Ending request: urn:request:5761b8a8-cb5a-45a5-868e-7e37347b2841 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:5761b8a8-cb5a-45a5-868e-7e37347b2841', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:25:20,383 [287] [DEBUG] [app] Ending request: urn:request:b60b9ecd-e280-4472-80e4-12d3ae5176a5 (/v1/_internal_ping) {'endpoint': 'v1.internal_ping', 'request_id': 'urn:request:b60b9ecd-e280-4472-80e4-12d3ae5176a5', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/v1/_internal_ping', 'path': '/v1/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': 'd44d4fab', 'user-agent': 'python-requests/2.31.0'}
gunicorn-registry stdout | 2024-06-14 05:25:20,384 [282] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:25:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
gunicorn-registry stdout | 2024-06-14 05:25:20,384 [287] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:25:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:25:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:25:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.001 162 0.001)
gunicorn-web stdout | 2024-06-14 05:25:20,384 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:25:20,384 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /v1/_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:25:20,385 [271] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:25:20,385 [269] [DEBUG] [urllib3.connectionpool] Resetting dropped connection: localhost
gunicorn-web stdout | 2024-06-14 05:25:20,387 [269] [DEBUG] [app] Starting request: urn:request:4f5c548e-3192-43e3-9910-b9a3b4909c8c (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:25:20,387 [271] [DEBUG] [app] Starting request: urn:request:bce5caf5-6c74-4a4f-910b-5eceadd87d70 (/_internal_ping) {'X-Forwarded-For': '127.0.0.1'}
gunicorn-web stdout | 2024-06-14 05:25:20,388 [269] [DEBUG] [app] Ending request: urn:request:4f5c548e-3192-43e3-9910-b9a3b4909c8c (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:4f5c548e-3192-43e3-9910-b9a3b4909c8c', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:25:20,388 [271] [DEBUG] [app] Ending request: urn:request:bce5caf5-6c74-4a4f-910b-5eceadd87d70 (/_internal_ping) {'endpoint': 'web.internal_ping', 'request_id': 'urn:request:bce5caf5-6c74-4a4f-910b-5eceadd87d70', 'remote_addr': '127.0.0.1', 'http_method': 'GET', 'original_url': 'http://localhost/_internal_ping', 'path': '/_internal_ping', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'python-requests/2.31.0'}
gunicorn-web stdout | 2024-06-14 05:25:20,388 [271] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:25:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
gunicorn-web stdout | 2024-06-14 05:25:20,388 [269] [INFO] [gunicorn.access] 127.0.0.1 - - [14/Jun/2024:05:25:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.31.0"
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:25:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
nginx stdout | 127.0.0.1 (-) - - [14/Jun/2024:05:25:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.31.0" (0.002 159 0.002)
gunicorn-web stdout | 2024-06-14 05:25:20,388 [271] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:25:20,388 [269] [DEBUG] [urllib3.connectionpool] http://localhost:8080 "GET /_internal_ping HTTP/1.1" 200 4
gunicorn-web stdout | 2024-06-14 05:25:20,389 [271] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:25:20,389 [269] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-06-14 05:25:20,393 [271] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:25:20,393 [269] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-06-14 05:25:20,397 [271] [DEBUG] [app] Ending request: urn:request:a0b36629-27ed-4566-8738-c62cd2b793e6 (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:a0b36629-27ed-4566-8738-c62cd2b793e6', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:25:20,397 [269] [DEBUG] [app] Ending request: urn:request:fb33f04a-75fe-4bea-8d68-82101425ca5d (/health/instance) {'endpoint': 'web.instance_health', 'request_id': 'urn:request:fb33f04a-75fe-4bea-8d68-82101425ca5d', 'remote_addr': '10.129.2.2', 'http_method': 'GET', 'original_url': 'http://10.129.2.44/health/instance', 'path': '/health/instance', 'parameters': {}, 'json_body': None, 'confsha': '778fe6a8', 'user-agent': 'kube-probe/1.28'}
gunicorn-web stdout | 2024-06-14 05:25:20,397 [271] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:25:20,397 [269] [DEBUG] [data.database] Disconnecting from database.
gunicorn-web stdout | 2024-06-14 05:25:20,398 [269] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:25:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
gunicorn-web stdout | 2024-06-14 05:25:20,398 [271] [INFO] [gunicorn.access] 10.129.2.2 - - [14/Jun/2024:05:25:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.28"
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:25:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.019)
nginx stdout | 10.129.2.2 (-) - - [14/Jun/2024:05:25:20 +0000] "GET /health/instance HTTP/1.1" 200 152 "-" "kube-probe/1.28" (0.020 118 0.020)
notificationworker stdout | 2024-06-14 05:25:21,591 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:25:21,591 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:21.592513+00:00 (in 0.000827 seconds)
notificationworker stdout | 2024-06-14 05:25:21,591 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:25:31 UTC)" (scheduled at 2024-06-14 05:25:21.591084+00:00)
notificationworker stdout | 2024-06-14 05:25:21,591 [82] [DEBUG] [workers.queueworker] Getting work item from queue.
notificationworker stdout | 2024-06-14 05:25:21,592 [82] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 25, 21, 592000), True, datetime.datetime(2024, 6, 14, 5, 25, 21, 592000), 0, 'notification/%', 50, 1, 0])
notificationworker stdout | 2024-06-14 05:25:21,593 [82] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
notificationworker stdout | 2024-06-14 05:25:21,593 [82] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:25:31.591084+00:00 (in 9.997910 seconds)
notificationworker stdout | 2024-06-14 05:25:21,593 [82] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:21 UTC)" (scheduled at 2024-06-14 05:25:21.592513+00:00)
notificationworker stdout | 2024-06-14 05:25:21,593 [82] [DEBUG] [workers.queueworker] Running watchdog.
notificationworker stdout | 2024-06-14 05:25:21,593 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:21 UTC)" executed successfully
notificationworker stdout | 2024-06-14 05:25:21,598 [82] [DEBUG] [workers.queueworker] No more work.
notificationworker stdout | 2024-06-14 05:25:21,598 [82] [DEBUG] [data.database] Disconnecting from database.
notificationworker stdout | 2024-06-14 05:25:21,598 [82] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2024-06-14 05:25:31 UTC)" executed successfully
exportactionlogsworker stdout | 2024-06-14 05:25:22,265 [66] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'exportactionlogsworker.py', 'pid': '66'}
manifestbackfillworker stdout | 2024-06-14 05:25:22,577 [73] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'manifestbackfillworker.py', 'pid': '73'}
namespacegcworker stdout | 2024-06-14 05:25:22,984 [75] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'namespacegcworker.py', 'pid': '75'}
gcworker stdout | 2024-06-14 05:25:23,100 [67] [DEBUG] [util.metrics.prometheus] pushed registry to pushgateway at http://localhost:9091 with grouping key {'host': 'quayregistry-quay-app-67b8967567-5cjxd', 'process_name': 'gcworker.py', 'pid': '67'}
exportactionlogsworker stdout | 2024-06-14 05:25:23,197 [66] [DEBUG] [apscheduler.scheduler] Looking for jobs to run
exportactionlogsworker stdout | 2024-06-14 05:25:23,197 [66] [DEBUG] [apscheduler.scheduler] Next wakeup is due at 2024-06-14 05:26:06.198336+00:00 (in 43.000479 seconds)
exportactionlogsworker stdout | 2024-06-14 05:25:23,198 [66] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:23 UTC)" (scheduled at 2024-06-14 05:25:23.197203+00:00)
exportactionlogsworker stdout | 2024-06-14 05:25:23,198 [66] [DEBUG] [workers.queueworker] Getting work item from queue.
exportactionlogsworker stdout | 2024-06-14 05:25:23,199 [66] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."queue_name", "t1"."body", "t1"."available_after", "t1"."available", "t1"."processing_expires", "t1"."retries_remaining", "t1"."state_id" FROM "queueitem" AS "t1" INNER JOIN (SELECT "t1"."id" FROM "queueitem" AS "t1" WHERE (((("t1"."available_after" <= %s) AND (("t1"."available" = %s) OR ("t1"."processing_expires" <= %s))) AND ("t1"."retries_remaining" > %s)) AND ("t1"."queue_name" ILIKE %s)) LIMIT %s) AS "j1" ON ("t1"."id" = "j1"."id") ORDER BY Random() LIMIT %s OFFSET %s', [datetime.datetime(2024, 6, 14, 5, 25, 23, 198345), True, datetime.datetime(2024, 6, 14, 5, 25, 23, 198345), 0, 'exportactionlogs/%', 50, 1, 0])
exportactionlogsworker stdout | 2024-06-14 05:25:23,205 [66] [DEBUG] [workers.queueworker] No more work.
exportactionlogsworker stdout | 2024-06-14 05:25:23,205 [66] [DEBUG] [data.database] Disconnecting from database.
exportactionlogsworker stdout | 2024-06-14 05:25:23,205 [66] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2024-06-14 05:26:23 UTC)" executed successfully